Visualizing 3D Geomodels using AR on mobile devices

Basic idea

The idea is to take 3D geological models (TIN models, either Layercake or BREPs, digital outcrop models, ...) and place miniature versions of them somewhere in the real world, e.g. on a table, and inspect them from all sides.

Since looking at a model with a friend is roughly 1500% cooler, collaboration features would be really nice to have, i.e. a second or third device can see the geomodel at exactly the same location, size, orientation etc. For a more in-depth look at this, see the Expandability section.

In general (and what I've learned from previous hackathons) I want to start small and expand the project as progress is made. This project in particular seems to be incredibly modular and would allow for a number of features to be added on individually without compromising the general "success".

As always, the main goal is to write code, learn something new and have fun!

Tech

I personally have a background in iOS app development and this will be the place where I intend to start this project. This includes: Xcode (macOS), Swift, ARKit, SceneKit.

In order to get the collaboration between devices going, Googles ARCore will be used to sync so called CloudAnchors between devices. Model state might be synced via a service such as Firebase (which is used by ARCore anyways I believe).

Apple is rumored to introduce their own way of syncing model state between devices on this years WWDC. This might be an option as well (google uploads the whole AR scene to their servers in order to sync model state, Apples version might only support two devices at once, but keeps the date between them). Of course only if no Android App would be developed.

I posses one iPhone that is capable of AR.

Source control will most likely be git. Open sourcing the code is possible as well.

Data

I will be able to bring some 3D geomodels with me. I convert them to wavefront objects (obj.) and hope to go from there. As I've heard, .dae objects are the way to go nowadays (which can be created from .obj files). But as always, more data is always better. A tool to convert SKUA .ts files to .obj files is in the links.

Expandability

As mentioned before, there are a lot of ways to expand this project. Here is a list of ideas (that is by no means finalized):

  • Android App: Because of ARCore, we would be able to sync model state with Android devices as well. A bit of research showed that the only thing to figure out would be which rendering engine to use. Something like Unity comes to mind.
  • Web App: It seems that it is possible to work with AR on the web as well. This is a topic I am totally clueless about, but I wanted to add it here anyways.
  • Manipulate the model and sync model state. Possible options are: highlight certain parts of the model, show an exploded view, add names to certain model parts, hide/view model parts ...
  • Website that handles the model state: Set up a website that allows a user to modify all parameters of the model state, hide/view certain models, highlight certain parts etc. This would emphasize an educational aspect as well. Think about a class of pupil looking at a model with their mobile devices and the teacher putting the emphasize on certain models/model parts.
  • Input data itself: I only have limited skills concerning 3D modelling and design (graphics related, not regarding geoscience). I bet it is possible to make these models literally and figuratively shine. In the end, this is an important aspect of visualization as well.

Skills that come in handy (basically a TLDR)

In order to work on the ideas mentioned in Expandability, or just in general, these skills would come in handy.

  • iOS development (Xcode, Swift, ARKit, SceneKit)
  • Android development (Android Studio, Java, Kotlin, ARCore, rendering engine e.g. Unity)
  • Exp with Firebase as a means of syncing model state between devices
  • Web development (Web AR, website to modify model state)
  • 3D modelling and design (Blender, .obj, .dae)

I think that the initial hurdle to get something going (i.e. visualize a 3D geomodel via AR on one device) should be reasonable low and leave enough time for the countless possibilities of expansion (if the data is more or less compatible). Collaboration between multiple devices would be my number one priority as it bears the greatest potential. Besides, development of Android or Web Apps as well as a website for modifying model state are completely independent from each other. This should make this project quite flexible.

If you are interested just send me a message on slack (bjoern) and we'll get talking :) (btw this project still needs a name, preferably including a pun concerning AR and geosciences, I have yet to think of something ...)

URLs of relevant information
https://developers.google.com/ar/discover/
https://github.com/stdonn/SKUA2OBJ
https://developer.apple.com/arkit/
https://firebase.google.com/
https://gitlab.com/groups/subsurface_hackathon_GeoARt/
https://youtu.be/F6vxMWG9aSg