Step 1: 3D Scanning.
Follow the
scanning Instructions
to record a 3D scan (.ply).
Make sure that the z-axis is up (or choose the correct up-axis below) and make sure that the scan is at metric scale, i.e., the point coordinates should be in meters.
The .ply should be less than 100 MB, include color and vertex normals. The scene should be metric scale, i.e., the units are in meters.
Our model is trained on indoor scenes (appartments, hotels, offices), this is where it will perform best.
The performance will likely be reduced on other scenes, for example outdoor scenes.
Instructions
Here we provide instructions for recording 3D scans using the "
3D Scanner App" for iPhone or iPad with a Lidar sensor (3D meshes recorded with other devices are also welcome). For privacy reasons, please do not record people, remove any personal objects from the scene, and only record on locations where you have permission to do so.
Step 2: 3D Instance Segmentation.
Upload your 3D scene (.ply) below to obtain the semantic instance segmentation.
Terms and Conditions
Any person submitting a scan to the Mask3D demo is required to assent to the following terms and conditions. The submitter is an original author of the scan, or a proxy acting on behalf of an original author. If the submitter is signing and accepting the terms on behalf of another individual, the submitter represents and warrants that the submitter has been granted the authority to make the submission by that individual and that the submitter’s acceptance of the terms binds that other individual.
Data Protection
The uploaded data is recorded by the submitter or is acting as a proxy on behalf of an original author.
The data will be made available to ETH Zurich and its collaborators for use and processing. In particular, the data is used to test and improve our Mask3D model on the task of semantic segmentation as well as future models and tasks.
The data is uploaded anonymously. They are therefore already anonymized at the time of transmission. This means that ETH Zurich and its collaborators have no knowledge at any time of the origin of the data, the location depicted or the ownership of the objects and locations depicted. If scans with such information are uploaded nevertheless, they will be completely or partially removed if recognized.
The data in the possession of ETH Zurich and its collaborators will be stored on the servers of ETH Zurich until the completion of the Mix3D project and any further projects, during the period relevant to the subject area, and if necessary, duly destroyed within the legally prescribed period.
The data can be used for further research activities within ETH Zurich and among its collaborators. The data may also be published in the form of a scientific publication in scientific journals, and international conferences. Further, the data can be passed on in its original and processed form to third parties for research purposes.
By uploading your scans you agree to sharing this data in particular for demonstration purposes at the 3DV event and for being released among the research community to facilitate research and development of indoor 3D scene understanding.
Representations and Warranties
The submitter makes the following representations and warranties:
No persons may be depicted on the scan. If scans with persons are uploaded nevertheless, they will be completely or partially removed if recognized. If the objects in the scans can be used to draw conclusions about people, ETH Zurich and its collaborators assume no liability for any damage to the personal rights of the person depicted. If scans with such information are uploaded nevertheless, they will be completely or partially removed if recognized.
Only scans from your own living area may be uploaded. If scans are nevertheless uploaded from other locations, prior permission has been obtained from the owner of the location, with reference to the purpose pursued.
Grant of the License
By uploading the scans, the owner grants the copyright of the scans to ETH Zurich and its collaborators. By doing so, the owner expressly consents to the use and publication of his data for scientific research among the research community. In addition, the owner grants ETH Zurich and its collaborators the right to process the data and to use this processed data in turn, as described above. In case of publication or disclosure to third parties by ETH Zurich, the source will not be acknowledged as the data is transmitted anonymously. For the same reason, the owner's assignment of copyright to ETH Zurich and its collaborators cannot be revoked. The rights granted will move with the Mix3D demo if the Mix3D demo moves to another steward.
By uploading your scans you agree to the above terms and conditions.
Step 3: Results.
For the processed scans, we provide:
- Visualizations
- Predictions (ScanNet FileFormat).
Selected Examples
Selected Failure Cases
Mask3D is currently limited in its ability to identify object classes that were not encountered during training.
Examples of such classes include futons (Japanese beds).
as
