3D and 4D World Modeling: A Survey

Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.

Learning through interaction with an environment, receiving rewards or penalties to learn optimal behavior policies.

In RL, the learner or decision-maker that takes actions in an environment to maximize cumulative reward.

In RL, the world the agent interacts with, providing states, accepting actions, and returning rewards.

A learned model of environment dynamics that can predict future states, used in model-based RL.

A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.

A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.

The field of AI that gets computers to extract meaning from images and video: what is in them, where it is, and how it moves.

Predicting distance of objects from the camera using monocular or stereo images.

Creating 3D models from 2D images using geometry and deep learning.

Creating new images from scratch or from text descriptions using generative models (GANs, diffusion models, VAEs).