AR Bookmarks - Dev Notes #5 - User Experience

Published on

What is a good user experience for an app where you place URLs around your room? It’s a good question and one that I’ve been thinking about a lot. On one hand I want to make AR Bookmarks as a way to play around with Swift, iOS development and ARKit. On the other-hand, it feels pointless making it unless it is at least half decent.

So I spent a while working on the user interface while trying to explore the various ways users may want to interact with the app. Currently to place a bookmark you just point and it goes a specific distance in front of you, so an obvious improvement there would be to be able to pin it onto a wall or table. I’ve also thought about how feasible it would be to pin bookmarks to objects, I’ve always loved the way Skylanders used to work where you would place a character on the portal and it would appear in the game. It makes a lot of sense to be able to for example pin a book’s website or source code to it, so every time you point your phone at the book no matter where you are it is always there in both the real world and virtual. However I knew I need to be realistic or I will give up on the project as I have done on countless of side projects before it!

Instead of perfection in an ideal world, I decided I needed to start small and focus on small improvements not the big ideas. So for now I’ve kept placing bookmarks the same but have developed on how to pick the URL. The way it works now is you point at where you want to place the bookmark, click anywhere on the screen then get a popup that lets you add a new URL, choose existing or cancel. If you add a new URL then you get a text box to type/paste into, then click a button and it appears (or you can go back to the menu). If you choose existing then you get a list of URLs (I call it the browse screen) that have been shared from Safari to choose from, you can also delete them by swiping. I figured this was a good place to start in terms of a UI, I tried to keep it as simple as possible. It gave me a good place to work from while thinking about the UX.

Small improvements

It was important to fix the things that were already starting to get on my nerves, because how can I expect anyone else to even attempt playing around with the app if it annoys me?

  • Added pull-down refresh to the browse existing bookmarks list
  • Made is so that the URL entered could be more flexible, for example allowing deep-links and made “example.com” work instead of requiring “https://example.com”
  • T... URL can now be blank, which acts the same as pressing cancel. Beforehand it would crash the whole app
  • Added proper navigation, took me a while to figure out how Navigation Controllers work when using fullscreen ARKit but got there in the end, now the browse screen has a back button that allows you to go back without choosing a bookmark (before you had to choose then delete it)
  • Changing from portrait to landscape now works as expected, instead of adding a large amount of whitespace
  • When you click to add a bookmark, it now uses that location instead of wherever you are pointing once the bookmark is finally added.

Placed or Unplaced

The more I used AR Bookmarks the more I realised it felt quite silo’ed, one part of the app allowed me to enter a URL and the other part allowed me to see URLs shared from Safari. The functionality on both sides was really useful but it felt like they didn’t go together very well. So what I’ve done is add a segment control at the top of the browse screen that allows you to view placed or unplaced bookmarks. This means you can see what bookmarks have been placed in AR, as well as those that haven’t. It brings the two sides together as you can now view bookmarks entered manually in the list. The big benefit of this is being able to view those bookmarks without being in the location they were placed. It also means that when a bookmark has been removed, it will go to unplaced until deleted, which can be useful if you just want to move it somewhere or use later.

I don’t want AR Bookmarks to be feature-complete as a bookmarking tool, it’s not made to replace other bookmarking apps with an AR gimmick, I want the location-based bookmarks part to be fundamental to the app. But there is definitely a lot of upsides to being able to see bookmarks from anywhere, such as to check for duplicates etc.

I considered how this would work, in some ways it makes sense to move the add URL screen to a button in browse. But for now I’m going to leave it as it is, as it feels quite nice to have the option instead of being instantly chucked into a different screen when you click to add a bookmark. I also considered the opposite, where you have to go through the browse screen before you can choose where to place it… but for now going to stick with choosing location first.

Share Extension

I’ve written before about the intricacies of the share extension, I originally wanted it to open AR Bookmarks to place in the world straight away but that’s not seemingly possible. So instead I started by using the default post screen. This felt really shoddy, more like tweeting than bookmarking.

So I looked at an app I like, Notability, and how they handled it. It seemed that they used a custom view, so I tried doing the same and made good progress. I got to the point where it would automatically add the bookmark to the app, and let you close the view. That still felt a bit forced though and as I’m new to iOS development I was quite unsure about all the auto-layout etc. The result didn’t feel very native.

Eventually I went back to it and it was one of those days where after ages of wracking my brain it just made sense. So I ripped out all the custom view stuff and replaced it with a UIAlert, the same that I use for placing bookmarks in the main app. So now it is a really simple process: you are on a page you want to bookmark, you click on share and “Add to AR Bookmarks” then it says “Would you like to add this page to AR Bookmarks?” with the option of yes or no. There’s currently no error handling but it seems that if the thing being shared is not a website, it will just fail silently. And because there is no server involved, clicking Yes feels just as quick as clicking No.

Next Steps

It is starting to feel more solid now, less like a prototype. But I can’t shake off the feeling of it being a bit weird having links floating around the room, so I want to improve how they are placed. Especially as it is quite common, when the device doesn’t have enough data about the room, to find the links moving around slightly as you look at them. It’s really disorientating and as the main purpose of the app, it needs to be as accurate and polished as possible. So my next step is to attempt to pin the bookmarks to the walls or surfaces, and possibly to do what I mentioned earlier about object recognition (that may be harder than I hope though).

    Read more about AR Bookmarks.