Karen Peng is a software engineer / NYU ITP 2015 alumnus. Originally from China, Karen followed her passion for art, code and the intersection of the two, to the U.S. to attend NYU ITP. She’s dedicated in building webGL for open source project P5.js, and making contribution to the creative coding community. She is currently a software engineer by trade, out of work, she’s consistently involved in data visualization projects, digital arts, and painting.
Chelly Jin (CJ): Describe yourself for individuals who may not be aware of your work?
Karen Peng (KP): Aloha, Karen here! I took some interesting routes and stumbled into art and code. I'm very glad and honored to be invited for this interview. I make interactive web apps that people could explore and play with, some of them are selected by Chrome Experiments. People find joy and inspirations while playing with them, and that is my biggest motivation of keep building things.
Here’s one of my projects, it’s called MotionEmotion. It’s a collaboration with my awesome classmate Jason Sigal at ITP NYU.MotionEmotion is an emotion & gesture-based arpeggiator and synthesizer. It uses the webcam to identify user’s emotion by facial expression, and the points of user’s movement. It draws triangles between the points of motion, and each triangle represents a note in an arpeggio/scale that is determined by the user's emotion.
Long story short, let me share the secret process of its development:
1. Idea exploration.
The idea of this project was originated from one of the experiments I did at my tiny room in Brooklyn. I’m very passionate about Augmented Reality and was exploring human movement and visual enhancement.
2. Rapid prototyping and collect feedback. I was lucky to take the opportunity of Music Hackathon and partnered with Jason to further develop the potential of combining movement, visual and music. We presented a prototype to the audience at the hackathon.
3. Further implementation, share it to the world!
After the hackathon and thanks to many friends that helped, me and Jason developed the project as a emotion and gesture based visual and music creation tool. Below is a performance that Jason gave to a class at ITP.
Isn’t it fun? Haha nah, there’re a lot of sleepless debugging nights you don’t get to see here. I’d say it’s bitter sweet, just like a lot of wonderful things in the world :)
CJ: A lot of your work is within the realm of code as a web developer and software engineer; where did your interest in the design and arts start to come into play?
KP: Software engineering is my job now, but I started programming from design and arts!
When I was a kid, I never had a chance to program and I thought computer was mysterious. Instead, I spent a lot of time doodling. I was major in advertising in college later. One thing caught my attention in advertising festivals was that there were a lot of works applied with new technologies, those works exceeded my imagination. Digging into that, I stumbled into the ‘creative coding’ realm, and it’s a new frontier! I found eyeO festival, Processing, Professor Casey Reas and Professor Daniel Shiffman, there’re pioneers worked in this area and there is still so much room for exploration among technology, design and art! After teaching myself basic coding, I applied for a program in NYU ITP -- Interactive Telecommunications program, and eventually flew myself to the Big Apple from Guangzhou China my hometown.
At that time I had no idea I would fall in love with programming and became a software engineer! ;p
CJ: What has your experience been like in integrating the two fields in the perspective of a software engineer?
KP: Design thinking methodology highly influences my day-to-day engineering work. It’s difficult to build something complex and perfect at once, what I often do is to build something small and testable, collect feedbacks for improvement, and keep rapidly iterating.
Art is always my source of inspiration to keep exploring. Mingling with my may or may not work ideas is my biggest hobby. There are a lot of failed or abandoned AR, VR, data visualization, I-don’t-even-know-what-that-is projects sitting in the chips of my laptop. Oh imagine they all come to live when I am sleeping lol.
CJ: What was your first experience with coding like? And how did you overcome hardship and difficulty?
KP: I created 4 bouncing balls on the screen. They blew my mind. I can’t believe it took me 23 years to discover this. Coding isn’t that intimidating, how come it has a big barrier to entry? Imagine your grandma, your 3-year-old niece, or anyone regardless of who they are, what they do, could learn to do that. Programming is powerful, and it should be ACCESSIBLE as well.
Whenever I have difficulty in coding, I bang my head against a wall. Haha nah, although it’s true to some degree, I’m just kidding… Luckily there’re a lot of resources online. Also if you are attracted to visual, Processing/P5 is an excellent tool to help you visualize the thought process. It also has an awesome community supporting the technology and its users. In addition, I found it super helpful to break down a big problem into many small ones. If you solve a small problem, you are one step closer from solving the big one! You could go get ice cream (very important), come back and tackle another one, keep moving!
CJ: You also worked on the WebGL renderer for p5.js: Why did you choose to work on WebGL? What were some of the toughest challenges and greatest rewards of working on WebGL?
KP: I want to draw a full circle to my journey. I started to learn programming through Processing, and it’ll be great to contribute back to the Processing community.
I was taking a webGL class and saw the potential unlocking the GPU. P5.js does not have webGL functionality yet, therefore I wrote and submit a proposal about integrating webGL with P5.js to the Processing Foundation. And it was approved. During the summer of 2015, I worked with my mentor Kevin Siwoff, designed and implemented a light-weight webGL renderer for P5.js.
The biggest challenge was to keep syntax consistent with Processing. In Processing 2d, you draw a rectangle every frame. However recreating and re-rendering a 3d geometry is very expensive. After several discussions and numerous tests, I figured out a way to build a caching mechanism, so that users could follow the Processing syntax yet have good performance. Killing two birds with one stone is my biggest reward ;) Here’s the P5.js webGL tutorial.
CJ: Where do you see the future of open source with art + code?
KP: It took me a long time to discover art + code, I hope others won’t have the same problem again! Open source will definitely increase art + code’s accessibility to people, what’s more, open source also supports a community to form and grow, which is beneficial for everyone in the long term. Best of all, I made a lot of friends along my P5 journey!
Furthermore, I think the future will be open source X + code. As a non-English speaker, I have English lessons since 4th grade. I can imagine programming becomes the new English. It will be a ‘must-have’ skill for anyone no matter who they are / what they do, because programming is crucial and already penetrates into every aspect of our daily life.