3D Design to AI: A Little Help, Please?

NeRF a 3D design savior or just another disappointment?

Finally. A 3D model from 2D views? (Picture from NeRF, Representing Scenes as Neural Radiance Fields for View Synthesis, Ben Mildenhall,  et al, paper presented at ECCV 2020)

Finally. A 3D model from 2D views? (Picture from NeRF, Representing Scenes as Neural Radiance Fields for View Synthesis, Ben Mildenhall, et al, paper presented at ECCV 2020)

The promise of computers to help with design is a promise yet unfulfilled. It
was a promise first made with the beginning of computers, starting with Ivan Sutherland’s Sketchpad in 1961[i],
the first program to make lines on the screen using a GUI (graphical user interface). It was primitive, drawing straight lines between points
and easily laughed off by veteran draftsmen as an expensive Etch A Sketch.

Computers became better at describing a design with solid modeling in the 1990s but still weren’t capable of designing, that creative process that can synthesize a shape from
thin air.

Designers, engineers and architects learned how to model in 3D. But what of all the designs that had been created since the beginning of drafting, and existed
as 2D, the traditional front, top and side views drawn on vellum. Computera: a little help, please?

Computers did help — by taking a scanned drawing and creating a CAD drawing, a process that came to be known as raster to vector conversion. Computers weren’t infallible
with their raster to vector conversion and designers still had to clean up CAD files after a conversion.

With the advent of 3D modeling, we asked for more help. Could computers take the front, top and side view and make the 3D shape from the 2D drawings or the 2D CAD files?
But the computer said “No.”

 It was not our first disappointment.

We got our hopes up again recently with the promise of generative design.
Here was a technology that could generate hundreds of designs in the time it took us to do one, designs we never would have thought of, lighter than anything we would have dared suggest. But generative designs were hardly ever manufacturable, except with 3D printing,
which was one imperfect technology to the rescue of another. Even if we had a 3D printer, the parts were wispy, fragile…

 It was yet another computer disappointment.

But hope springs eternal in the engineer’s breast. Enter AI and the promise that the computer can learn from our successes and failures. Here was machine learning based on the premise that given this you get that, and with enough examples of this to complete its training, the computer will be able to provide the answer to any problem.

What would an astronaut riding a horse on the moon look like? There may be no request too strange for the DAL-E 2 image creator which uses AI to conjure images based on natural language prompts. (Picture courtesy of DAL-E.)

What would an astronaut riding a horse on the moon look like? There may be no request too strange for the DAL-E 2 image creator which uses AI to conjure images based on natural language prompts. (Picture courtesy of DAL-E.)

And it worked. IBM’s Watson beat Ken Jennings, Jeopardy champion. There were chess champions, go champions… bested by computers. AI got more ambitious.
Why not detect tumors from X-rays and MRI scans? Or credit scores based on your payment history. Here is the English translation of just about any language you are likely to encounter, thanks to the largesse of Google. Here is DAL-E that promises to create any picture given semi-English commands.

But can it be used seriously for product design? For architecture?

Sure enough, says Stephen Coorlas, a trail-blazing architect that is using Midjourney to create some of the most fantastic 3D designs we have ever seen.

But what of usable 3D models from 2D images from sketches, drawings, photographs?

I can do that, says NeRF, a new AI player. NeRF is not to be confused with Hasbro’s Nerf, used for kids’ footballs that won’t break windows or bullets that won’t put eyes out. NeRF stands for neural radiance field.
It is another technology that can turn photographs into 3D rendering.

To be continued…


[i]Sketchpad, James Pyfer, Brittanica.com.