One of the questions we get a lot is “How do we make AR instructions if we don’t have 3D models?”
It’s a valid question. The WorkLink platform was built primarily around the concept that organizations would be leveraging their own products’ CAD models to create augmented reality training and instruction materials. Many of our clients are using it in exactly that way, and having no difficulty in achieving that workflow. If you’re in that category then congratulations! you can probably grab a coffee.
The scenarios where this approach doesn’t fit tend to be in a few general categories
- “We need to assist our employees on equipment that is supplied by a vendor”
- “The CAD files exist and we own them, but we are struggling to get them released to us”
- “This equipment pre-dates our CAD software”
In actuality, most of these scenarios are likely to be short-lived. Where IP protection is a concern, for instance, CAD files can be converted and simplified at source to maximize the value to instruction while minimizing the exposure of proprietary information. In addition, the very nature of self-authoring keeps that exposure limited to your internal content authors and a pre-approved workforce working across a secure network. As the benefits of AR instruction and assistance become more commonly understood, these barriers are starting to fall.
In the meantime though, it can be extremely useful to have techniques for these situations, and we thought we would share a few, as well as publish a WorkLink project specifically made with no supplied or ‘made to order’ 3D content whatsoever as an example. We chose a basic car maintenance example, commonplace and straightforward, but also a good reference point for more complex situations.
LESS IS MORE
One key thing to understand is that good AR instruction is really about adding as little to the user’s workspace as possible. While movies tend to portray augmented reality as the ability to add as much as possible, the fact is that this doesn’t work well. Our goal is to provide small, but key, additions to the space which will have maximum impact. From this perspective, having complex 3D models of the equipment is actually not beneficial at all. When working on an engine after all, the engine is there. We have no need to reproduce it. For a large variety of processes, arrows, circular beacons, basic tools and simple shapes are all that is needed to communicate everything your user needs to know… Particularly when they are animated effectively and placed exactly where the user needs them. All of these things, along with the ability to place video and images, are provided for your use in the WorkLink platform. For common objects that aren’t included, support for standard file formats makes adding 3rd party content (from public websites etc) a simple process also.
CONTEXT IS KING
Under these circumstances, the AR author is still left with one significant challenge. You start your project, secure in the knowledge that you a combination of simple content is more than enough to communicate exactly what your end-user needs to know… as long as they are placed accurately in the workspace. Without a model of your equipment in the scene, how can you place your content? You need a reference framework of some kind… context.
There are a number of strategies for establishing this framework. One method is to take some key measurements and create some simple 3D shapes to represent key landmarks in your work area. This can be effective for straightforward situations, and if you have ready access to your equipment, some trial and error may be an acceptable approach. If the area you want to present instructions in is basically flat (or a series of flat spaces) such as a control panel for instance, it might also be an option to take photos (carefully, and square to the camera) and bring those images into your project as stand-ins. For more complex, demanding projects, it may be worth the effort to create 3D objects that are more representative of the actual equipment. 3D models can be created at various levels of detail, and there is ample middle ground between detailed CAD models and simple shapes. Although this skill set isn’t available in every organization, it’s also not particularly challenging or expensive to access. For some projects it may be worth the relatively small expense of generating some models for this purpose. This method is particularly important if your process demands an extended disassembly or assembly process, where layers of parts are needed.
For circumstances where the area is more complex, or where access is more challenging, what’s needed is some form of reality capture. This term covers a broad variety of options, but the essence is basically the same… the ability to go into a space and quickly generate a 3D model of it without any particular skills. These models can be extremely useful for providing context, but you will not have the ability to ‘disassemble’ them. These types of models will represent a contiguous surface with no recognition of where one object ends and another begins. Great for providing a reference framework, so you can use it as a map for placing your instruction, but you will likely not show this type of model to your end user.
Here are some of the major options:
Laser Scanning: If you have access to laser scanning equipment, or your budget allows contracting these services, this can be an effective way to get a surface model of a work area.
Photogrammetry: This is relatively simple process, requiring access to a camera and. Essentially the process is to take a large number of photographs (>100) of a work area, from a wide variety of angles and distances, and using generally inexpensive 3rd party software to generate a textured 3D model. Results can vary, and depending on the software you may have to manually scale the resulting model, but this technique can be quite useful in the right circumstances.
Depth Camera/3D Sensor: This is currently our preferred method. Utilizing a handheld depth camera, either built into a smartphone, or as an external accessory to a tablet or smartphone, you can essentially walk around an area and generate a simple textured 3D model ‘on the fly’. Formats used are compatible with WorkLink, so you can bring the model in immediately and use it as a quite accurate reference for placing content.
Results from all of these methods can provide workable results, but detail levels vary. The goal here is to allow a rapid reference framework to be put in place, low detail levels are entirely acceptable for the less expensive approaches.
If you’re interested in this approach, I highly recommend you check out our “A3 Maintenance Demo” using the free WorkLink authoring app. It’s designed to take full advantage of the Microsoft HoloLens, so if you have access to one, definitely use that, but you can download it on any device’s store. Log in as a guest and load the A3 project, then either use a standard Scope AR marker or “Interactive Mode” (on handheld devices) to view it. You can also see this project featured in the video at the top of this post.
The project includes a series of maintenance instructions designed to be viewed directly on the vehicle itself. We’ve included some additional content strictly to help demonstrate the concepts discussed here. The car outline is a commercial 3D model, but is included only to provide context for those viewing the instructions away from the car, and would not otherwise be needed. The engine model itself was scanned in about 15 minutes using a smartphone with a 3D depth camera. Again, when viewing these instructions on the vehicle itself, this model would not normally be included. We’ve included it in the demo to show what type of results can be expected from this sort of process, and also to help viewers understand the context of these instructions.
Visibility switches (blue spheres) are provided to allow you to show and hide the various models. Turn off the car body and engine to view the instructions as they would appear when seen on the real car.
As you will see, this approach makes for a very effective style of instruction. For many of our clients, projects like this are the answer to a difficult question, allowing them to quickly create effective instructions without the need for a lot of engineering support or external resources.
For more information on creating AR Work instructions with no coding or previous experience, check out the WorkLink page, or see our Youtube channel, and be sure to keep track of the latest Scope AR news on Facebook, LinkedIn and Twitter with the links below.