Nvidia’s NeRF neural network creates 3D models based on several photos

Nvidia has introduced a new version of the artificial intelligence tool Neural Radiance Field (NeRF). It is adapted to work with the latest video cards with tensor cores, on which you can create 3D scenes based on two-dimensional photos in a few milliseconds. The previous version of NeRF required several hours to achieve a similar result.

The principle of operation of NeRF has not changed, it undergoes express training on several images of the object from different angles, and then combines the data and interpolates them to fill in the gaps. The output is the most realistic three-dimensional model that can be used in various scenes and compositions. The new version uses the technology of encoding a hash grid with multiple resolutions to optimize work on graphics chips.

As Nvidia explains, speed is the main quality of the new neural network, because they are going to use it in autonomous driving systems. It will be easier for the autopilot to calculate the distance to the interference if he sees it in three-dimensional form, and it will be easier for robots to navigate in the real world. In addition, NeRF is sure to find applications in entertainment, education and architecture.