doUmind is a new Android-based app that converts hand-drawn mind maps into digital versions and then enables you to export them to a number of popular formats. This magic is performed using powerful AI software that has only recently become powerful enough to master this formidable challenge.
But two French engineering students, Virgile Garnier and Juliette Breurec, decided to take it on and have created an simple, elegant approach to transforming hand-drawn visual diagrams into computer-based maps that can be modified and improved – and save their creators the hours that would be required to do this task manually.
I recently interviewed them to learn more about doUmind – how they envisioned this remarkable tool, how it accomplishes this remarkable task and what’s next for it.
Chuck Frey: Where did you and Juliette come up with the idea for doUmind?
Garnier and Breurec: We were both engineering students specializing in mechatronics at the end of our studies at our university, IMT Mines Alès. We both simultaneously did 2 years of French pre-engineering undergraduate studies for competitive entrance exams to National Engineering Schools, then we entered our school the same year. That’s how we met each other and developed strong friendships with a whole group that was very supportive in the creation and development of doUmind.
Frey: What inspired you about the potential of transforming hand-drawn mind maps into digital versions?
Garnier: For one of our first-year courses at our university IMT Mines Alès (third year studying engineering), the teacher asked us to work in teams of about 8 on a subject; and the assignment had to carried out on a mind map. So the mind map was drawn on paper, everyone added ideas, it was a very productive brainstorming session. Fourty-five minutes later we were satisfied with our work, but we couldn’t hand the draft we had worked on together to the teacher.
So after searching in vain for a mobile app or software capable of scanning everything, one of us spent 2 hours reconstructing everything in order to get an aesthetic and clean result in PowerPoint. It struck me that the “intelligent” part of the work only represented a tiny portion of the invested time, which made this experience very frustrating. I then wrote down this start-up idea in a notebook but I didn’t have the skills to start such a project.
Breurec: In our 4th year, we had to take an advanced course of our choice and I convinced Virgile to choose the Inferential Mathematics course, which dealt partly with artificial intelligence, because I knew it was interesting to learn more about the subject, regarding the idea he had told me about.
After talking to our professor, he gave us some documentation and we had free rein to build the neural network (artificial intelligence) that met the needs of doUmind. Two months later, our algorithm being very powerful, we were all the more enthusiastic about creating doUmind when we realized that the technological challenges were not as unbeatable as we had imagined.
The concept of doUmind also evolved a lot at that time. Although the idea came from mind maps, we had the technology (for recognition) to process any kind of diagram, and we were thinking of developing a more general application.
But, looking at the market, we realized that mind map enthusiasts were much more dynamic and demanding. This allowed us to go back to the initial idea of making an app specialized in mind maps. Mind maps having a codified structure, we could also develop specific layouts and thus offer much more elaborate designs than for random diagrams!
Afterwards, we went to meet with the administration, believing that a gap year would be appropriate to develop a project of this scale. IMT Mines Alès immediately accepted and, encouraging its students’ initiatives, provided us with all the support and resources necessary to help us discover the world of entrepreneurship. We therefore began in 2018-2019 and used our self-taught spirit to cover all aspects of the creation of doUmind.
Frey: Why do you think that no one has tackled this challenge before?
Garnier and Breurec: It could be that artificial intelligence developers are working on other types of projects which are more financially promising.
Artificial intelligence applications are multiplying but, as every innovation, it starts with the biggest market before spreading to the narrowest subjects. So we can expect this kind of technology to take time to reach niches like mind mappers.
Frey: Who does doUmind appeal to? Who is your ideal audience?
Garnier and Breurec: Everyone uses post-it notes at some point. Sometimes it’s to remember or learn useful information, other times it’s to structure a thought process, or to successfully lead a meeting by making sure of getting everyone’s input.
We believe that the use of mind maps is as powerful and diverse. And we hope by making the process of making mind maps to communicate ideas easier, we can appeal to a large variety of people. But we knew that the first audience we would reach would be mind map enthusiasts such as teachers, managers or students.
Frey: When did you launch this app?
Garnier and Breurec: We launched it back in June 2nd, 2020. In less then a month, we were happy to get more than a thousand users only on android!
Frey: Wow! What type of feedback have you received from users so far?
Garnier and Breurec: Most of them were very enthusiastic and the rest were especially helpful because they pointed out their needs. Thanks to their feedback, we were able to add the most requested features to enhance the app’s user experience.
Frey: What problems does doUmind solve for its users?
Garnier and Breurec: The main setback in mind map building is the amount of time put into it. The use of mind maps seemed to be only accessible to a well-informed audience and the step to fully understand and use this tool well is a difficult one to take. We hoped that by making this transition process easier we could make mind maps more appealing to a large variety of people.
So technically speaking, our main goal is to help people save time in the mind map creation process without interfering with their habits. An experienced user can save time with doUmind and get back to his favorite editing tool, and a inexperienced user can get a fully editable, very aesthetic digital mind map in less than a minute, even if they never made one before.
Frey: How does it convert shapes and handwriting into a digital mind map?
Garnier and Breurec: All the heavy lifting is done on our server. Since the users take a picture, we have to handle all the problems associated with it: the shades, the background, the uneven brightness, etc. So the first step is to enhance the picture the best we can to highlight the important information and distinguish it from the useless data such as a shadow, a background color or a grid pattern, borders.
From there, we have the useful data. Then we find everything that could be a bubble and we use an artificial intelligence engine we built and trained to know whether those are real bubbles or not. From there, we find the links and build the whole mind map. In a different program, also running on the server, we use another artificial intelligence program to recognize text. Because we now know where the bubbles are, we know where to look for them.
Similarly, a third program running on the server receives the mind map data and converts it into a beautiful design, accordingly to the user’s choices.
Frey: How accurate is it at translating user scribbles into digital map content?
Garnier and Breurec: As you can imagine, the recognition relies heavily on the quality of the picture as well as the drawing. We added some enhancements to our AI program to maximize the quality of our recognition, such as avoiding shadows, drawing bubbles and links needing to touch the shapes they connect.
Our program is able to work fine even if some of these criteria are not followed, but it is capable of producing a perfect recognition if the picture complies with most of them. There are some examples on our YouTube channel showing how well it can perform even if the drawing or the picture are not perfect.
Frey: It looks like the app does its best guess at how a word is spelled, but then enables the user to correct it. Is that right?
Garnier and Breurec: You can always edit everything. We know our automated recognition is not perfect and will never be, just like any scanner. After all, it is a machine trying to understand a human. So there is an integrated editing tool at every step of the process.
If the recognition finds a non-existent shape, you can delete it. If it missed, one you can add it. If you decide you want to add a bubble that was not even on your drawing you can also do that. Same goes with links. You can even ignore the picture and build everything from scratch.
Frey: Has the technology to do this type of handwriting and shape recognition gotten dramatically better during the last several years?
Garnier and Breurec: As far as shape recognition goes, it could have been done with the technology from years ago but, indeed, it’s getting better and more accurate as computers are becoming more powerful. As you might know, machine learning relies heavily on processing enormous amounts of data to “learn” so the power of today’s computers is critical to the development of this technology.
Handwritten text recognition is therefore still barely existing since it’s very difficult for a computer to know where a character ends. So instead of the simplicity to make a machine learn a hundred letters, it has to be trained to recognize millions of words in several languages. Only Google was able to access that amount of data when it asked millions of users everyday to decrypt captchas with barely readable texts.
Frey: I noticed that doUmind offers a surprising number of export formats. Why so many and how did you choose them?
Garnier and Breurec: Initially, we developed a PowerPoint format because it is the most used software for diagram building and presentations. But the audience we have reached so far is mainly made up of mind map enthusiasts, who are used to their mind map editing software. They told us it would be a real time-saver and much more interesting if we developed those formats.
So thanks to their feedback we were able to provide them with Xmind, MindManager, Freemind and text formats.
Frey: Are there any limitations to doUmind that the readers of my blog should know about?
Garnier and Breurec: For now, our app only recognizes hand-drawn mind maps on light backgrounds. We made a choice to allow only diagrams that strictly comply with the mind map codification. In other words, because the user’s input is a picture, it has also to comply with some rules to make the recognition successful.
We are considering making another app allowing other types of diagrams, with a new program capable of optimizing the bubble positions accordingly to a diagram with no strict codification. For example, detecting if bubbles are intended to be aligned horizontally or vertically or if lines are crossing.
Frey: I understand that an iOS version is under development. When do you think that will be released?
Garnier and Breurec: This is really hard to say. iOS apps are made with a completely different code. So even thought we don’t have to recreate the program running on the server, we have to build all the iOS user interface from scratch. Moreover, iOS development requires us to work on a Mac, so we need to buy one. We are considering to launch a crowdfunding campaign to help us with this process.
Frey: What’s next for doUmind?
Garnier and Breurec: doUmind is only a month old and has actually already evolved a lot! On June 2nd doUmind didn’t recognize text, only offered PowerPoint as an editable export format, was only in English, the recognition had significant problems with high-quality pictures and special symbols in the texts were sometimes misunderstood.
We still have a lot of ideas. We already mentioned some of them like the capability to scan something else than mind maps or the iOS development. Both will be big projects requiring some time. In the short term, we are now trying to improve the text recognition.
You can learn more about this innovative app on the doUmind website.