AI will soon help make 3D models out of 2D. It’s in the lab.
The conclusion of our 4-part series on AI at Autodesk with head of R&D at Autodesk, Mike Haley. Part 3 is here.
Mike Haley: Our customers, advanced and not so advanced, are asking us what we are going to do and how we are going to help them AI and not just with large language models. You’re never going to build a large model that is intelligent enough unless it gets enough data. What made large language models possible was the web. The web is full of things to learn from. There is no web for 3D—at least not at the scale that we need. Those models are with our customers. The challenge over the next 5 years is, how do we start creating large enough datasets across multiple customers across multiple industries so we can train these systems to start being predictive? Those are the kinds of things we’re exploring right now. Most importantly, how [can we] do that ethically? How do we use that data in a way that is transparent and creates value from data directly for our customers? We’re not Facebook. We’re not trying to sell you something because we saw something in your data. We’re trying to give you a tool that is going to make your workflow better. But to do that, we’re going to have to understand the patterns in your digital data.
Roopinder: I understand that you respect your customers’ IP. But is there a way to generalize or anonymize that data so you could still use it? Also, there is publicly available data that is probably quite complete, like the U.S. Trademark and Patent Office, which has an incredible amount of data. There are model libraries, such as 3D Warehouse….
We already use lots of that stuff. Onshape published a library some years ago. We published one 2 years ago, an open library. We took the whole Fusion gallery dataset and packaged it up and sent it out there. Most of the work that’s happening, like mine, and in academia, is being fed by exactly that approach. But even that is not at the capacity that we really need. We need several orders of magnitude past that. The other thing that makes AI powerful is because it looks for patterns and variance in the data. I could say ‘I trained this great. I gave it 100 million models. So, it’s going to be great after looking at 100 million models of bicycles. By the end of the day, it’s probably going to do pretty good at bicycles. It’s going to be worse at anything else. That’s obviously an extreme case. But this is the point of variance: the more variety you see in that data, the more the model is going to fit to the possibilities of what you’re thinking about and creating. With creating, it’s not just about the quantity; it’s about finding that variety across the industries, different domains, different problems, datasets, different stages of design … all of that is going to be critical.
The other thing I’ll mention to you is a conversation I started having with a bunch of friends about what data everybody sharing immediately. You could do it like Tesla [which is training on customers’ data, mentioned in Part 3 of this interview], but what if what if it’s safety on construction sites? Everybody wants to solve that problem. Companies will want to share safety information because everyone cares about safety. There are categories of information for which we imagine consortiums of companies would come together and say, “Let’s just solve safety across construction,” for example. “Let’s throw all of that information about safety issues and what caused them and mitigate them all together.” Let us make a SafetyGPT.
Another big thing could be sustainability. Everybody’s responding to sustainability goals right now. We could have something that sorts out materials and design approaches that could lead to lower environmental impact businesses and more sustainable solutions.
Over the next couple of years, we’re also going to see this materializing as people understand more of the benefit of creating large datasets and training these AI models. I think there will be more openness to saying, “Let’s go and do this.”
Some years ago, I set up our internal ethics practices as well as auditors as sort of a side gig because I honestly believe that our customers need, and the industries need, to feel that they can trust their software partners. Frankly, if I was in their shoes, I would be suspicious. Maybe they think we would replace stuff. I think there is a new way for companies and businesses to function now. The ones that are going to respond most effectively and work most transparently with their customers. We will push them. In some cases, we will push our customers, saying. “Hey, we’re going to do this. This is a new thing. It’s coming. It’s here. Even if you don’t want it, it’s going to happen. We’re going to be transparent about it. We’re going to work with them on it. We’ll find the right balance, build trust. This is central to all technology companies right now, even outside of our business.
At last, let me make a request. It’s personal. We are doing a home remodel. We hired an architect. I was hoping for a 3D model. What I got was paper drawings. We could not find a single architect in our area that did 3D. We studied the drawings. I’m getting the picture. My wife is not. I’m thinking, “Why isn’t there a way to automatically take the drawings and make a 3D model. I used to teach CAD. I used to have students take isometric views and turn them into 2D views and vice versa. That was 30 years ago. The more I learn about AI, the more I wonder, “Why can’t AI do that?” We certainly have all the datasets we need to go back and forth between 2D and 3D models. I know a lot has been done with automatic 2D view creation, but the reverse, 2D to 3D, still remains to be done. That would be quite popular, in my opinion.
The good news is we have that working in the lab. We have it working right now as we speak. If you are sketching and you want it to be made into a 3D object or you start with an iso view of an object and you want that to be made into a 3D model. We have been able to start doing that. But also, as you request, going from blueprints, an actual drawing, and pulling it back into the model, kind of extruding it into the third dimension—create the objects, create the right associations … we have that working in the lab. There still are some limitations, but in principle, yes, we have that.
That would make you a big hit with Revit users. They wouldn’t have to create 3D models from scratch from their trove of 2D drawings. They would be most of the way towards a BIM model.
I’ve met many Autodesk customers whose job was just that, taking old blueprints of buildings and making BIM models from them because they are required for approvals, or remodeling [or design reuse on the next project].
I don’t think AI will take away jobs as a whole, but that is one job that it should.
That is a perfect example of automation. There is zero creativity involved. It’s just a pain in the ass.
You’ve got a lot to do. I better let you go. I said I had one last request, but I’ll leave you with one more: a natural language UI for CAD. I maintain that CAD is difficult to learn and use. Each time I have to learn one, it’s hard. I get bogged down with simple commands. And here I am, immersed in CAD all my professional life. I am really looking forward to a better UI. That’s it. Good luck to you. I will be looking for great things to come out of the lab.
Great talking and seeing you again, Roopinder.
