eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Google has a number of research projects underway aimed at making computers smarter and technically versatile. One of those projects involves teaching machines how to draw.
On April 11, Google researchers released a technical paper describing “sketch-rnn”, a neural network that has been trained by using thousands of crude human-drawn images to construct basic drawings of its own.
One of the goals of the paper is to show that machines can be taught to draw certain things, like the sketch of a house, a tree or a dog, in a manner similar to humans.
“As humans, we do not understand the world as a grid of pixels, but rather develop abstract concepts to represent what we see,” wrote two of the papers authors, David Ha and Douglas Eck, who are researchers with Google Brain, the company’s deep learning research group.
People have the ability to visually depict what they see using a short sequence of strokes. For example, a child sketching a house invariably tends to depict it with a triangle on top of a square with a door and window added to it.
While such a simple drawing may not resemble reality exactly in the way a photograph would, it can be useful in communicating what an individual might be trying to represent. “Objects such as man, woman, eyes, face, cat, dog, etc. can be clearly communicated with simple drawings,” Ha and Eck said. Similarly, it is easy to convey an individual’s emotions through line drawings and a few strokes, they said.
The effort is to see if computers can be taught to do the same thing using basic strokes. Some potential applications include tools that can help artists by suggesting multiple ways to finish a sketch for instance or tools that can help pattern designers generate similar but unique designs for wallpapers or textiles. The technology could also be useful in an educational setting, such as for teaching people how to draw or to improve on their drawings, the two researchers said.
In order to train its neural network, Google used a set of crude hand drawn sketches submitted by users participating in the company’s Quick Draw A.I. Experiment. Quick Draw is a site were users are asked to draw specific objects, such as a hand, a lobster or a rake—in less than 20 seconds while the neural network tries to guess what the user is doodling.
Users then have the choice of submitting their sketches to Google where it becomes part of a larger body of data that is used to train the network.
The neural network that Google described in its technical paper this week contains of 75 image classes, each containing some 70,000 samples of sketches submitted by users via Quick Draw.
According to Ha and Eck, the model that Google has described in its paper is capable of reconstructing sketches submitted by humans on its own. Even when fed with sketches that have been deliberately messed up—such as a cat with three eyes—the model is able to discern the mistake and generate an image of a similar looking cat but with two eyes instead.
Some of Google’s work in this area has already begun to produce results. One example is a web-based tool that Google released this week called AutoDraw. The free tool uses machine learning to recognize what a user might be attempting to doodle on their phone, tablet or computer and then automatically completes it, or offers suggestions for completing the doodle.