KIT 724 Virtual and Mixed Reality Technology Assignment 2 - AR

Share this post:

Introduction

Nowadays, augmented reality application attracts broad attention in daily life. Augmented Reality apps are software applications that merge digital visuals such as image, audio, and video content into the user’s real-world environment. The AR software is widely used in training, working and entertainment purpose in various industries including education, healthcare, tourism, and marketing (Lee 2012).

The application is based on the idea of enabling users to draw on 3D real-life models (objects) instead of 2D to better engage younger audiences as well as simulating practical design challenges and providing real-time visual feedback. the application can be useful for various stages of education such as drawing and sketching classes as well as professional uses.

Screen recording capture of the FREE Drawing application

Figure 1

Description of the application

My application is named “FREE Drawing” which is used as both a drawing and sketching class tool and a professional design app helping users to better recognise the real-life impact of their drawing. Users can project brushes and drawings in the digital world onto real-life objects. The app provides them with the experience of a real-life colouring book where you can draw textures on the actual object itself.

FREE Drawing Application would be designed to provide a strong foundation for long-life learning abilities, and offer a suitable teaching method combined with users’ activities to facilitate a users' drawing skills and future success. The advantages of the interface in the “FREE Drawing” application are categorised into the following aspects.

The application has an interface that enables the user to interact with reality. To be more specific, users can draw textures of different colours, and patterns on to object that they have chosen.
The app enables the user to learn how to draw/apply textures to 3D surfaces with curvatures and corners. This learning can later be applied to professional skills such as packaging design, make-up design, tattooing etc.
The interface is user-friendly to ensure users can use simple gestures to manipulate the app resulting in a better user experience.

Interaction Design

Interactions are essential to the application. The idea is that the user can draw on the digital screen and the end result will be projected onto the object via screen and digital camera. A user can click on the buttons in the application to trigger interactions with the application to scan, draw and view the project drawing.

The “FREE Drawing” Application demo is created with an interface by using augmented reality with a web camera and computer screen, which is shown in Figure 1.

The main interactions of the “FREE Drawing” Application include that:

Users can take a 3D scan of a real-life object by clicking the scan button in the application and rotating the camera around the object based on the direction given. This model is then used as a reference to map the drawing onto the actual object.
Users can then paint/draw on the 3D model using the live drawing mode, where they would point the camera to the object and have their drawing projected onto it in real-time.
They can also use model drawing mode, where they can draw on the scanned model without facing the camera towards the object to have better postures. In the model drawing mode, they can also move the camera to different angles around the object.
There are different brushes, colour palettes and editing tools available to use similar to a traditional drawing and photo editing application such as photoshop.
The final drawing can be projected at any time by selecting the final artwork file, clicking view and pointing the camera to the object.

Storyboard


user takes a 3D scan of a real-life object	the object shows on the screen

user can draw/paint on the object	The final drawing can be projected at any time by selecting the final artwork file, clicking view and pointing the camera to the object.

Technical Development

As the design of the application is very ambitious and I do not currently have enough technical expertise and time to finish a polished application, I’ve created a few simple models as a prototype for the final project.

The models are created in Unity and used as an example of a scanned real-life object. Users can use the free draw tool to draw simple coloured lines to a 2D texture mapped on this model.

This 2D texture map is what will eventually be used to project onto the real-world item.