Can technology and data bring back to life one of the greatest painters of all time?

The Next Rembrandt is a new painting created from historical Rembrandt data.

On Tuesday morning, The Next Rembrandt was unveiled in Amsterdam: a 3D printed painting, made solely from data of Rembrandt’s body of work. Thus bringing the Master of Light and Shadow back to life to create one more painting. Only this time, data is the painter, and technology the brush. 

A portrait of a man in black 17th-century clothing with a white collar and a hat has been unveiled in front of a packed auditorium. The painting was created using data from Rembrandt’s total body of work using deep learning algorithms and facial recognition techniques. The portrait consists of over 148 million pixels, based on 168,263 painting fragments from Rembrandt’s oeuvre.

Blurring the boundaries between art and technology, this artwork is intended to fuel the conversation about the relationship between art and algorithms, between data and human design and between technology and emotion.

The story behind this unique painting is explained in full detail on www.nextrembrandt.com. After the launch, The Next Rembrandt will be on public display at a location that will be disclosed at a later date.

The portrait was created through a highly detailed and complex process (involving a team of data scientists, developers, engineers and Rembrandt experts), which took over 18 months and resulted in 150 gigabytes of digitally rendered graphics. The project is a cooperation between presenting partner ING Bank, advertising agency J. Walter Thompson Amsterdam, supporting partner Microsoft and advisors from Delft University of Technology (TU Delft), The Mauritshuis and Museum Het Rembrandthuis.

It all started with the data. The first step to making The Next Rembrandt, was analyzing all 346 of Rembrandt’s paintings using high resolution 3D scans and digital files, which were upscaled by a deep learning algorithm. Supporting partner Microsoft contributed their cloud platform Azure to host and analyze this data, which formed the basis for The Next Rembrandt.

Then came determining the subject. The majority of Rembrandt’s entire collection is made up of portraits, which is also the most consistent subject.

Together with Rembrandt experts the demographic segmentation of the people in these portraits was defined. Ending up with the subject: a portrait of a Caucasian male between the age of thirty and forty, with facial hair, wearing black clothes with a white collar and a hat, facing to the right. 

After this, the subject’s features were generated in the style of Rembrandt. A software system was designed that could understand Rembrandt based on his use of geometry, composition, and painting materials. A facial recognition algorithm identified and classified the most typical geometric patterns used by Rembrandt to paint human features. It then used the learned principles to replicate the style and generate new facial features for the painting.

Next, these individual features were assembled into a fully formed face and bust according to Rembrandt’s use of proportions. When the 2D version of the painting was ready depth and texture were added. With the help of TU Delft, a height map was created to identify patterns on the surface of canvases. By transforming pixel data into height data, the computer could mimic the brushstrokes used by Rembrandt.

Finally, to bring the painting to life, an advanced 3D printer that is specially designed to make high end reproductions of existing artwork was used. In the end, 13 layers of UV-ink were printed, one on top of the other, to create a realistic painting texture.

All this has resulted in a work of art that portrays the power and the beauty of data and technology and that will fuel the conversation about where innovation can take us.