New Chapter at Banjo Digital

I’m happy to share the news that I’m joining up with Banjo Digital, an AR/VR firm in Austin that creates experiences for simulation/training, education, product design, and marketing. I’ll be serving as Technical Director, and will be starting next week. I’m excited about the move and look forward to doing great work together there!

Using Swagger APIs with Unity

February 6, 2018 Update: I figured out a workflow with the experimental .NET 4.6 support in Unity to actually make this work with Daydream, and have updated the workflow here. This solves the annoying caveat that I’d previously posted.

Swagger (aka OpenAPI) lets you define API structure in a machine-readable way. This allows all kinds of cool functionality: automatic docs, code generation across languages, etc. Here’s the workflow we’ve devised for consuming Swagger APIs in Unity. (Note: I’ve tested this on a Mac, but haven’t tried it on a PC yet.)

  1. Load your Swagger specification into the editor at https://app.swaggerhub.com/. (The easiest way is just to copy and paste it into the editor window.)
  2. Verify that the auto-generated documentation on the right looks correct.
  3. Choose “Csharp” from the “Download → Client” dropdown.
  4. Unzip the downloaded csharp-client-generated.zip file.
  5. Open a terminal window.
  6. Go to the directory of the new unzipped download. (e.g. “cd ~/Downloads/csharp-client-generated/“)
  7. Build your DLL: /bin/sh build.sh or build.bat if you’re on windows.
  8. Your DLL is now in the “bin” subdirectory of the downloaded folder and is called “IO.Swagger.dll”. Copy this to your “Scripts” folder in Unity.
  9. Find “Newtonsoft.Json.dll” (packages/Newtonsoft.Json.10.0.3/lib/net45/Newtonsoft.Json.dll) and copy it to your “Scripts” folder in Unity.
  10. Find “RestSharp.Net2.dll” (packages/RestSharp.105.1.0/lib/net452/RestSharp.dll) and copy it to your “Scripts” folder in Unity.
  11. You’ll also need to round up the DataAnnotations DLL from somewhere in your Unity package (Unity.app/Contents/MonoBleedingEdge/lib/mono/4.5/System.ComponentModel.DataAnnotations.dll) and copy it into your “Scripts” folder in Unity.
  12. Unity loads the DLL and makes the IO.Swagger namespace available to you.

Using it:

Take a look in the docs folder of the project you downloaded from the Swagger editor. It includes nice Markup files with C# sample code documenting the APIs in your new DLL. The sample code makes a great starting point for accessing the DLL’s functionality.

Great Big Obnoxious Caveat:

This doesn’t currently work when running on Android due to this bug in Unity’s .NET implementation. We ended up doing a last-minute rewrite to use Best HTTP instead, which was a crying shame. Hoping this bug will indeed be fixed in the future and make this workflow viable on that platform.

 

Getting Unity and Arduino Working Together

Yesterday I set myself a goal of getting Unity talking to Arduino, the microcontroller that’s hugely popular in the maker community. I was interested in doing so because several of our VR projects have been site-specific installations that could benefit from a large LED scoreboard, physical actuators (rumble motors, heat lamps, fans, etc.) to heighten the experience, or just a “This person flailing his arms around can’t see you; please stand back!” warning light.

Fortunately, given than both Unity and Arduino are very popular, this path is fairly well-trodden already. The most common approach is to establish a serial connection between the computer running Unity and the Arduino board. This is straightforward to do, though it requires tweaking a few settings in Unity to allow access to the serial libraries.

For the Arduinos that have USB port, the physical connection is as simple as plugging in a USB cable. For the boards that have only serial pins, wiring in a converter chip will be necessary to bridge the two. I used the Arduino Uno since it supports USB and I didn’t feel like soldering.

The next decision is whether to write your code from scratch or to use a library to ease this task. If you’re more of a DIY person, the amount of code you have to write isn’t outrageous. Alan Zucconi has an excellent article that will guide you through what you need on both systems.

If you’re looking to get up and running quickly, however, you may prefer one of the options available in the Unity Asset Store. These come with all the code you need for many use cases already written, a variety of examples, and niceties like custom UI in the Unity editor. I ended up using Marc Teyssier’s excellent Uduino package, which has good documentation and supports digital input and output, analog input and output, and servo motor control out of the box.

(I ended up having trouble at first, as my board wasn’t setting the pin mode to digital output when the code commanded it to. I wasn’t sure whether this was a problem with my board or with the code, but adding some additional commands to the Arduino code to make sure that the pin mode got reset cleared up the problem.)

I rigged up a simple relay circuit, created a sample scene, and before too long, had a light in my living room turning on and off with the scene lighting in Unity. Viola!

This is merely a proof of concept — a technical spike to sort out some unknowns. Now that we’ve got these waters mapped, I hope to add some production physical effects to future projects. Excelsior!

Moving to VR

About a year ago, I decided it was time for a career shift. I managed a fabulous team of mobile developers who were a joy to work with, but I missed creating things. I talked to my boss at Mutual Mobile about the problem; he encouraged me to chart out a way back to an engineer role within the company.

I thought carefully about my options. I could go back to iOS engineering, which my previous boss had done. He was delighted with the change and had no regrets whatever about stepping down from management to an engineering job. But I enjoy learning and figuring out new things and, while iOS is a fantastically enjoyable platform to develop for, iOS 10 seemed pretty mature. I moved from web development to mobile because the web stopped feeling like the wild west. Now I had the same sense about mobile.

So I looked at alternatives. Mutual Mobile had a months-old Virtual Reality practice at that point that was gaining steam. While I hadn’t yet spent any serious time with Unity, which the team uses for its work, my son and some of my fellow engineers both had and were clearly enjoying the experience.

I floated my plan to retool myself into a VR engineer to my boss and the VR team, all of whom were gracious enough to give it an enthusiastic thumbs-up. So in early 2017, I began a self-administered crash course in Unity and VR. I found Unity’s online training materials to be excellent, and by the time we executed my official transition to the VR team, I was able to jump in and contribute to the team without a problem.

(I was still green compared to the other two excellent engineers working on our projects, but made up for it by continuing to do management of our engineering and project management teams.)

I’ve now gotten to contribute to a Cardboard project and do all the engineering for an Oculus Rift project. (The former should be available in the mobile app stores soon, and the latter will be installed in the Bass Pro HQ in Springfield in a few weeks.) The team’s also doing work with several other really interesting clients; I’m excited about the work ahead.

Now that I’m past the initial hurdles in Unity, I’m learning things I think might benefit other developers and/or VR enthusiasts. Accordingly, I’m going to post things here to document my learning. My areas of particular interest include: Good Software Architecture in Unity, Cool Stuff our Team is Doing, Using Physical Props in VR, and Accessibility in VR.

If you’re interested in any of these areas or just in creating VR, please get in touch or just follow along. I’m excited about this next phase of my career, and will be happy to have company!