9 Lessons Learned in Rolling Out Dimagi’s Mobile Device Management Pilot

November 5, 2018/ adaweb / French / 13,288 comments

Enumerators, who are people like you and I, like to download Candy Crush, Facebook, YouTube videos, mobile screen savers, to their mobile phones, and use up significant battery life and data on those apps. This can be frustrating to a project and push them way over budget and deadlines.

In response, earlier this year, Dimagi launched a mobile device management service to help improve mobile data collections surveys with better management controls for mobile devices.

Dimagi created an MDM tool called MobileAtWork for the international development space and ran a pilot with ten organizations testing MobileAtWork in various geographies, with various types of enumerators, project managers, and administrators, and in various development settings.

Our pilot did not go as planned.

9 Lessons Learned From Our MDM Pilot
In the spirit of sharing our success, and failures, here are the 9 lessons learned in rolling out our MobileAtWork tool at Dimagi. We hope you can learn from our mistakes – we sure have!

1. Know Your User’s Technology
We started by limiting the Android OS requirements to OS 5.1 or higher, which ended up being a huge barrier to entry for most of our clients. A lot of them are operating on older devices that can’t upgrade past OS 4.4, so it was an immediate non-starter.

Unfortunately, the feature set we wanted to test out was only possible to access on newer devices, so we made the strategic decision to stick with 5.1, and move forward with a smaller set of pilot participants.

2. Work Within Project Limitations
For those who had the right OS, enrolling devices into the MobileAtWork product was tough. We built it with the assumption that IT administrators and project managers would have access to their devices, and could scan a QR code to enroll any given device into MobileAtWork.

This was a huge barrier to overcome as it turned out, because many projects had devices already in the field. Our enrollment method made it nearly impossible for these clients to enroll their deployed devices, so they had to wait until a new deployment was happening or the entire team gathered in one location again, which often wouldn’t happen for months.

3. Expect Leakage During a Pilot
After this, there were just a small, heroic few projects left in the pilot! One org was able to enroll a handful of devices and deploy them back into the field with MobileAtWork installed for a few months. Another enrolled on a few devices in their office just to test the product.

I had expected that 10 out of 10 of the enrolled projects would use the tool, but that did not end up being the case.

4. Design with Everyone in Mind
The feedback was a mixed bag. Once we got past the OS and enrollment barriers, some people liked the tool and others did not. Specifically, front line workers and enumerators were not thrilled about the prospect of a hidden layer of information about their device usage being sent back to the head office.

Administrators and project managers were more excited about the product, but we weren’t collecting all the information they needed to make data driven decisions about their project.

5. Focus on User Feedback
There were potential solutions to these problems, but they went against our initial assumptions.

First, we realized the key would be to show front line workers their own data usage behavior inside the MDM app on their device. This could be a potentially big change from before when they could not see what information was being sent to their project managers. When we pitched this idea to users, it was met with a lot of excitement; front line workers would finally be able to see how their use of FB or YouTube was eating up their data, and control and change their own usage.

Second, we realized we should pass more granular data usage analytics back to the project managers and administrators. We realized that with a deeper level of data presented in a user friendly graphical way, admins could quickly make actionable decisions like, “Let’s message everyone about limiting time on FB.” or, “Mobile data usage on DHIS2 is much higher than we expected across our entire fleet; let’s increase the monthly data limit to accommodate everyone’s needs.”

6. Go the Extra Mile for Users During Pilots
We were able to build these features into the tool, but it took a lot of work with a very limited set of resources. In the end, we were able to remove the first two barriers to entry as well- so now there is no OS requirement (all Android OSes work) and there is no QR code to scan (you can enroll from anywhere by clicking on a unique URL).

We also made messaging much more intuitive and kept our location tracking as is, but now admins can narrow down the date range to check the data within a certain set of dates. We also added App Blocker into the app.

So, if you were to ask yourself, “I wonder how much data my team used last week, where the highest users were located during the peak usage times, and which apps they were using at that time,” you could answer that question in MobileAtWork now. You could also block that app on an individual’s device so that it no longer eats up precious and expensive data.

7. Simplify, Simplify, Simplify
One of the bigger challenges we had during this process was explaining the complexities of this powerful tool to our clients. In the beginning, we had two different enrollment methods that worked on different OSes and could give the user different sets of features in the product.

In fact, we had to write up long wikis and add decision trees to our tool just to help people understand how to use it. This was more complicated than building your first CommCare app! (jokes, jokes.) We realized we were losing interest from the first call because the tool sounded overwhelmingly complex, and we had to make it easier to use.

8. Add Value Immediately
Our biggest learning was to never underestimate the importance of user onboarding and early experience. In our case, the poor first experience for admins, project managers and front line workers trying to find viable devices, recalling them from the field, and then scanning QR codes, left people very frustrated off the bat, and a lot less willing to put up with (small and large) bugs later down the line.

The next big learning we had was to show immediate value to every user right away- by giving the front line workers a view of their data usage over the past month we were able to offer positive value to users who otherwise felt frustrated and nervous about an MDM tool on their device.

9. Always Be Iterating
Once we launched the second version of the tool, we’ve had some very successful stories come about. One client has been so happy with the product that they pitched their larger PM team and a second project has already started deploying it with their front line workers!

We’re now opening up MobileatWork at a discount to a small subset of viable clients over the next month. If you are interested in deploying this tool in your project, please reach out to me through mobileatwork.co.

By Shabnam Aggarwal, a strategic advisor at Dimagi.