"My World" Automatically Generated "Real World": British Exhibition AI Compensation New Technology

Origin: Machine Heart

Editor: Sanan

Minecraft Village self-constructed image of the world? English: A true world where AI comes to the rescue.

April 12th, 2021, in the GTC 2021 tournament in England, Huang Ren was removed from the exhibition. 3D screen automatic conversion to photorealistic screen technology (GANcraft). Recently, GANcraft's article has been submitted on arXiv, and we have understood this technical detail.

Be careful with GANcraft's AI, which is not a real world, but in the Minecraft world. Since watching the performance, the effect of the presentation has been amazing.

Previously, the use of English-language GPU enabled Tensor Core opening DLSS (Deep Learning Super-Advanced) technology, the image quality of many games, and the number of games greatly increased. As the technology of GANcraft matures, we expect to see more and more powerful 3D image refinement technology in the future.

GANcraft, submitted by Zekun Hao, a researcher at Yukong Naru University, is a kind of non-directive divine system, used for generating a large 3D block world ) image. The method is to create a language image element and import it, and distribute one label in each block, such as soil, grass, tree, sand, or water. Arithmetic General 3D world display is connected to the physical function, parallel to the real image. Outside of the visual field, GANcraft has a visual language and style.

Player Transformation Scene Designer

Given the past image super division rate calculation method is different, GANcraft The hope solution is "World to world problem": A fixed one piece with a language written in a square block world, like a popular game "my world" middle scene, GANcraft Noboru General is a shared partnership, but a biased and truthful new world. The new world can be created from any viewing angle, and the generated images are consistent with existing images and have realistic images.

GANcraft technology has greatly simplified the 3D construction process under the scene. Since then, this demand for multi-year industrial knowledge has now made it possible to create a 3D player for every "our world" toy city.

Usually, deep learning super-distribution rate calculation requires the original image of the truth, and the corresponding situation is progressing. Currently researching, author Yasho GANcraft based on 2D math training model (MUNIT, SPADE), based on 2D correction and 3D deformation generation image method wc-vid2vid, and from 3D matching math calculation intensive learning and progressing NeRF-W Comparing progress.

Comparatively, we can see im2im method (MUNIT and SPADE) legally realistic visual consistency, because of unlearned 3D construction, and each time is independently generated; wc-vid2vid can be generated The video is consistent with the video, but because of the lumpy number of body parts and training, the error caused by the difference accumulates, and the image quality progresses rapidly as time goes on. The effect of NSVF-W is approaching with GANcraft, but with a little bit of precision.

In GANcraft Generative Results, 3D Visual Consistency is guaranteed for God's use, and we have created a new reality that has never existed before.

How to see the AI ​​``indemnity'' in the real world

We have only one suitable physical condition God-made model, it can display the real world, we need a kind of method to proceed with the special training, we have to use what the real image is generated under the original image .

Under the circumstances of the existing reference image, the generation of the anti-network GAN is a small model, and the acquisition of the unlimited god-like dyeing task has been a little successful. However, due to GANcraft's application site, the problem is that the challenge is even more challenging -- given the truth of the world, "our world" has a completely different label distribution. For example, a certain landscape covered with snow, a desert completely covered with water. There is also a scene of various contents that spans within a small area. In addition, when viewing the model from the Goddess, we have a viewing angle distribution that is impossible for us, and a matching image that can be obtained on the Internet.

As shown in the above picture, because of the resilience of the task, use the mutual network open reference work to see the progress against the competition (first line) meeting results. Production and use of the truth is one of the main contributions of GANcraft craftsmanship, and it is possible to produce high-quality effects (second line).

Generation of ``fake image'' method is based on the use of SPADE model, which is divided into three versions, and generates a realistic image. When we divide the world from one block to another, the real image and the generated image from the same image are shared with each other. This is a little bit of a visual mismatch, and we have been able to use additional loss functions (for example, perceptual sum L2 loss) to improve and improve training.

Currently in GANcraft, the researcher has joined the 3D body dyeing device and the 2D image space dyeing device to display the real scene. The author's head is defined by one or more body elements to the world's God's Land: A fixed square block world, so that each block's angle distribution is a special learning amount, and the use of three-line method is possible within the body element. . After that, we can use the MLP-style definition of the launch site, its location number, parallel language, and its shared style content creation, generation point specialization, and its volume density.

This article, just need to revisit the fixed number, we can get a 2D special plan, which special plan will be the last to pass through.

Because of the easy-to-use MLP progress model, this GANcraft's second stage system structure can provide high image quality, simultaneous reduction of computational volume and internal occupancy, which is based on the computational bottle based on the complex system. GANcraft presents a system structure capable of processing an extremely powerful virtual world. Researchers are currently testing, and people can use a scale of 512 x 512 x 256, which is equivalent to 65 English or 32 football stadiums all over the world.

Finally, what is the sky? Since then, based on the physical divine dyeing method, the sky is an ``infinite distance'' sky model, but the sky is an important part of the real image. In GANcraft, the calculation method is added to the MLP against the sky, and the MLP projection angle direction is a special target, and its size is similar to the target point in the shooting range. After that, the special effects are completely opaque, and the residual transmittance mixed arrival element is based on the original.

GANcraft's generation process captures style images. During the training process, we need to use the actual image creation style for reference. During the evaluation period, we will allow GANcraft to provide different styles of images, subject to export restrictions.

Introduction video of GANcraft:

Is this the future of the game? There is a futuristic version of "My World", we can see this new technology application.

Review《GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds》

Thesis link: https://arxiv.org/abs/2104.07659

Reference content:

https://nvlabs.github.io/GANcraft/

Construction & Vision —— 2021 AI Online Conference

April 22, 14:00 - 18:00

Is there any multi-machine learning cost? Large-scale machine learning, how is the company's digital conversion model realized?

<

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< In-depth analysis of the science and technology industry's new culture, secret AI/ML how to help companies accelerate innovation.

Branch Hall 1: Machine Learning Practical Secrets

Branch Hall 2: Digitization of Artificial Intelligence Enterprises

Branch Hall 3: Road to Realization of Large-Scale Machine Learning

Branch Hall 4: AI Service Assistance Mutual Link Network

Branch Hall 5: Before the Opening

Branch Hall 6: Collaborative Intelligent Life

6 Oita venue, are you more interested in the topic?

©THEEND

Transfer