top of page

#Project 4# - Prototype & Evaluation

  • Team Gen Z
  • Nov 9, 2021
  • 23 min read

Updated: Jun 12, 2023


ree

Contents:

Introduction

In this project, we desired to update the interface of a job seeking website in Malaysia that is jobsmalaysia.com.my in order to help more Malaysians in need. In this phase, we will convert our storyboards into prototypes. We will evaluate our prototypes by conducting heuristic evaluation using Nielsen’s heuristics then improve our design. After that, we will set 3 different goals to be done by 5 random users with the prototype. Before that they will be given a short brief. We will watch and observe the condition they do the tasks as well as record their time taken. In addition, we will conduct an interview with the users after they tried out the prototypes. They will be asked some questions about their opinions and experience using the prototypes. This is to help us to improve our interface design.

Digital Photos of the Pieces of the Prototype


ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

ree

Briefing Note

Hey, welcome, and thank you for taking part in and contributing to this user testing. In this testing, you are participating to test out the high-fidelity prototype of a job searching website that allows users to search for jobs and post their jobs.

You must complete three distinct tasks. You must use the think-aloud method on each task. In this method, you must verbally state what you are doing and what your thoughts are while performing each task. Essentially, you must state your progress while performing the task, such as what page you are on and what step you will take to complete the current task. If you are jammed or unable to complete a task, you should state that you are unable to complete the current task and then proceed to the next task.

The first task, you are required to chat with the job poster to obtain more information about the job. You are first going to find a job post that interests you and chat with the person or company that posted the job.

The second task, you are required to discover suitable jobs to apply for. You are going to use the search function of the prototype to find a desired job post and apply for the job interview.

The third task, you need to post a job. You are going to use the post job function in the prototype to post a job and accept an applicant that applies for the job that you posted.

To summarise, it is critical that you accomplish the task while using the think aloud method. This is because the method will reassure us and provide us with details about the understanding of the user towards the prototype.

Testing with Users

User 1

Age: 35

Gender: Male

Education: Degree

Occupation: Software Engineer


Task 1


Observation:

The user does not have trouble finding the ways of message to job poster in our prototype. The user performs the task in half a minute by navigating to the function of chatting with job poster through the interactions our prototype provided. First, the user login and enter to the main menu which provides information and allows users to search or post jobs. The user then clicks on the search jobs button to enter the searching jobs section. Next, the user filters the job by searching jobs with keywords and location of the jobs. Then, the user clicks on a job post to enter the job post page and the user clicks on the chat bubble button to enter the chat page.


Result from interview:

1) Is it easy to use the system to do the task?

The system is easy to use to do the task as the interactions provided are clear and functional.


2) What are the difficulties when doing the task using the system?

The difficulty when doing the task using the system is the chat with job poster function is not obvious as the chat bubble icon is hard to notice.


3) Do you think the system helps you perform the task efficiently?

I do as the system provides the function of chatting with job posters.


4) Do you want to use this system in your daily life? Why?

I do not think so as I currently have a fixed job and there is not a need for me to search for a job using the system; However, it might be useful when I need a job.


5) What are the problems you find in the system?

The problem I found is the difficulty of noticing the chat bubble icon in the system and I expect the system to have a quicker way to chat with job poster except for navigating to the job post, then can chat with the job poster.


Summary from interview:

In general, the system itself can be used to perform the task of messaging the job poster perfectly; However, there exists a lack in terms of user interface design as the chat bubble icon is difficult to notice at first place. Besides, the function of chatting with job poster except for navigating to the job post before chatting with the job poster should be provided.


Task 2


Observation:

The user does not have trouble in finding the ways of discovering suitable jobs to interview and applying for the job in our prototype. The user performs the task in 45 seconds by navigating to the function of discovering suitable jobs to interview and applying for the job through the interactions our prototype provided. First, the user logs in and enters the main menu. The user then clicks on the search jobs button to enter the searching jobs section. Next, the user filters the job by searching jobs with keywords and location of the jobs. Then, the user clicks on a job post to enter the job post page and the user clicks on the apply now button to enter the application page. The user then fills in the application form and submit a file such as resume during the application and clicks the submit application button to submit.


Result from interview:

1) Is it easy to use the system to do the task?

The system is easy to use to do the task as the interactions provided are clear and functional.


2) What are the difficulties when doing the task using the system?

I do not have any difficulty when doing the task using the system as all the buttons provided are clear and understandable as texts describing the function are provided.


3) Do you think the system helps you perform the task efficiently?

I do as the system provides the function of helping people to submit applications to the job poster and wait for the response from the job poster.


4) Do you want to use this system in your daily life? Why?

I do not think so as I currently have a fixed job and there is not a need for me to search for a job using the system; However, it might be useful when I need a job.


5) What are the problems you find in the system?

There is no problem found when I am doing this task using this system.


Summary from interview:

In general, the system itself can be used to perform the task of discovering suitable jobs to interview and applying the jobs perfectly. The user does not suffer from any problem during testing the system by performing the task.


Task 3


Observation:

The user does not have trouble in finding the ways of posting and managing the job. The user performs the task under 1 minutes by meeting all the requirements in this task by using the interactions provided by our prototype. First, the user logs in and enters the main menu. The user clicks the post job to enter the manage post page. Then, the user clicks on the add new job button to enter the add new job page. The user then fills in the information of the job and clicks confirm to add the new job post. The user is then redirected to the manage job page and this page will allow the user to manage the job post the user just added. The user clicks on the “Manage” text and the information of the job post will be displayed and the user is allowed to review applications and view chats through this page. For reviewing applications, the user clicks on the review applications button to view the list of appliers. The user clicks on manage text to view the information, application form and attachments provided by the applier. The user clicks on the reject button to reject the applier or accept button to accept the application. Next, the user can either go back to review other applicants or go back to the main menu. The user clicks on review other applicants and clicks the back button to enter manage the job post page and start viewing chats with applicants. The user clicks on the view chats button, then clicks on an applicant to view or start chatting with the applicant.


Result from interview:

1) Is it easy to use the system to do the task?

In overall, the system is easy to use to do the task as the interactions provided are clear and functional.


2) What are the difficulties when doing the task using the system?

One of the difficulties when doing the task using the system is the “Manage” text when managing the applicants of the job is too small causing the function to be harder to notice and click on. Another difficulty is the inconsistent word used in the system. For example, some of the parts are using applicant while some is using applier to describe the person who applied for the job.


3) Do you think the system helps you perform the task efficiently?

I do as the system provides all the functions required in this task and all of them are functional.


4) Do you want to use this system in your daily life? Why?

I do not think so as I currently have a fixed job and I am not an employer. Therefore, I do not see the need for me to use the system in my daily life.


5) What are the problems you find in the system?

One of the problems I found is that the “Manage” text is too small. Another problem is the ways of viewing the applications and the chats are different. For example, for viewing applications, there is a “Manage” text provided for users to click whereas no button is provided for users to click when viewing the chats. And the last problem I found is the inconsistent word to describe the person who applied for the job.


Summary from interview:

In general, the system itself can be used to perform the task of reviewing and referring to the requirement, salary and benefits offered by other companies in the same state, posting and managing the job; However, there exists some problems found in the system such as inconsistent words and interactions provided by the system and some of the texts or buttons which provide functions required in the task is too small causing the interactions difficult to be conducted.

User 2

Age: 25

Gender: Male

Education: Diploma

Occupation: Software Developer


Task 1


Observation:

The user entered his details such as username and password. Upon successful login, he clicked on ‘Search Jobs’ and entered ‘Programmer’ in the job title field and ‘Johor Bahru’ in the location field. After that, he clicked ‘Search’ and job listings appeared on the screen. The user clicked on an interesting job listing and started reading the details. After reading through the details, the user still had a few things he wanted to know. He clicked on the chat bubble on top right and started messaging the job poster.


Results from interview:

1) Is it easy to use the system to do the task?

The system is easy to use because everything is straightforward and the terms and icons used are understandable by normal users.


2) What are the difficulties when doing the task using the system?

A small difficulty that I encountered was that the chat bubble icon is pretty small, and it took me a short while to find it.


3) Do you think the system helped you perform the task efficiently?

Yes, because I was able to chat with the job poster and ask questions without any issues.


4) Do you want to use this system in your daily life? Why?

I will definitely be using this system to look for my next job, but not use it on a daily basis because I do not need to look for a job everyday.


5) What are the problems that you find in the system?

Well, the only problem I find while doing this task is that the chat bubble icon is hard to notice at first glance.


Summary from interview:

According to the user, it is easy and straightforward to chat with the job poster. The minor problem faced by the user is about the chat bubble icon being too small and this affects the visibility of the icon.

Task 2


Observation:

The user once again entered his details such as username and password. Upon successful login, he clicked on ‘Search Jobs’ and entered ‘Programmer’ in the job title field and ‘Johor Bahru’ in the location field. After that, he clicked ‘Search’ and job listings appeared on the screen. The user scrolled through the job listings and clicked on one that he found interesting. He started reading through the details of the job. After reading through the details, the user decided to apply for the job. He clicked on ‘APPLY NOW’ and filled in the application form as well as attached his résumé in the attachments section. After that, he clicked on ‘SUBMIT APPLICATION’ to submit his application.


Results from interview:

1) Is it easy to use the system to do the task?

It is easy to look for suitable jobs and apply for an interview because I just need to login and search for jobs. The application process is also easy since I just need to fill in an application form and attach my résumé before submitting.


2) What are the difficulties when doing the task using the system?

For this task, I do not encounter any difficulty because all the options and navigation buttons match the real world, such as the ‘APPLY NOW’ and ‘SUBMIT APPLICATION’.


3) Do you think the system helped you perform the task efficiently?

Definitely, since I was able to search and apply for a job that I desire with ease.


4) Do you want to use this system in your daily life? Why?

As mentioned previously, I will gladly use this system when I am hunting for my next job.


5) What are the problems that you find in the system?

I did not encounter any problem in the system when doing the task of looking for suitable jobs.


Summary from interview:

According to the user, looking for a suitable job and applying for it is easy and he did not face any problem at all doing this task.


Task 3


Observation:

The user entered his details such as username and password in the login screen. Upon successful login, he clicked on the ‘Post Jobs’ button. Then the user added a new job post by filling in the details of the job. After that, the user clicked ‘Manage’ to manage the job post to review applications submitted by job seekers and also chat with users who had questions for the job poster.


Results from interview:

1) Is it easy to use the system to do the task?

Posting a job opening is easy in my opinion.


2) What are the difficulties when doing the task using the system?

There are a few difficulties that I encountered while posting and managing the job post. The ‘Manage’ button in managing job posts and applications are small and do not have clear borders around it which can confuse users whether or not that is a clickable option. Next, there is no option to filter or sort the applicants, which means I have to scroll through all applicants even though some do not have the required qualifications.


3) Do you think the system helped you perform the task efficiently?

Yes, the system helped me perform the task efficiently because I was able to post and manage a job opening without struggle.


4) Do you want to use this system in your daily life? Why?

I will not be using this system on a daily basis, but I will use it when I want to post a job opening.


5) What are the problems that you find in the system?

The problem that I found is that the ‘Manage’ button in managing job posts and applications is too small and less visible. Another problem I found is that there is no filter or sort function when managing applications so I could not quickly filter out candidates which are qualified for a particular job opening.


Summary from interview:

According to the user, it is easy to post job openings. However, the user did mention a few problems with the system. One of the problems is that the ‘Manage’ function in managing job posts and applications are quite small and not enclosed with borders which might confuse users whether that option is clickable. The second problem is that there is no feature to filter and sort the job applications, meaning that unqualified applicants are still visible on the list.

User 3

Age: 27

Gender: Male

Education: Degree

Occupation: Data Analyst


Task 1


Observation:

Generally, the user has no issue to contact with the job provider as the task is done in a minute. The user has to log in his/her account and click the “search jobs” to search for a job. Later on, the user enters the job title and location and clicks the search button. Then, after clicking the job post, the user can view the details and then proceed to the contact session by clicking the speech bubble for further inquiries.


Result from interview:

1) Is it easy to use the system to do the task?

Yes, the functions and interactions are very direct and user friendly.


2) What are the difficulties when doing the task using the system?

Generally, there are no difficulties faced in this task.


3) Do you think the system helps you perform the task efficiently?

Yes, this system somehow saves up my time to obtain information from the job provider.


4) Do you want to use this system in your daily life? Why?

Yes, I would like to use it as it is user friendly and cost effective which could be beneficial to those who are rushing for a job.


5) What are the problems you find in the system?

The icons used in the interface are advised to be enlarged a bit so that it is more obvious and easier for all the users to figure out.


Summary from interview:

In summary, the system almost fulfils the user perfectly. The system interface may require some update on the design to enhance the visibility so that the user can minimise the uncertainty while conducting the task.


Task 2


Observation:

Generally, the user has no issue to contact with the job provider as the task is done in a minute. The user has to log in his/her account and click the “search” to search for a job. Later on, the user enters the job title and location and clicks the search button. Then, after clicking the jobs offered, the user can view the details and then proceed to the job application by clicking the “apply job” button. After that, the user needs to enter their details and upload attachments such as resume followed by clicking the submit application button. A message will pop up to notify the user that their job application is successful.


Result from interview:

1) Is it easy to use the system to do the task?

Yes, it provides clear directions and I could do the job application in a minute.


2) What are the difficulties when doing the task using the system?

Generally, there are no difficulties faced in this task.


3) Do you think the system helps you perform the task efficiently?

Yes, this system somehow saves up my time to apply for a job.


4) Do you want to use this system in your daily life? Why?

Yes, it certainly helps me to apply for jobs especially when I'm out of a job.


5) What are the problems you find in the system?

There are missing notifications regarding the result of the job application which I may have missed out on the opportunity to get to work.


Summary from interview:

In summary, the system is able to let the user make job applications easily. However, there is a missing functionality which is to notify the user about the results of the job application. With the functionality mentioned above, the user will not miss out any chances for their desired jobs.


Task 3


Observation:

Generally, the user has no issue to contact with the job provider as the task is done in a minute. The user has to log in his/her account and click the “post jobs” to post a new job. Later on, the user needs to click the add new job button followed by entering the job basic information and details. Then, the user needs to click confirm so that the job is being posted. Next, the user clicks the “manage” beside the job post made by the user. The user then proceeds to review applications to manage the applications. Users are required to click “manage” that appears beside the applicant to view their application details and attachments to decide whether to accept or reject the application. After accepting the application, there will be a message telling the user that the applicant is accepted.


Result from interview:

1) Is it easy to use the system to do the task?

Yes, the system design is simple yet easy to use.


2) What are the difficulties when doing the task using the system?

Generally, there are no difficulties faced in this task.


3) Do you think the system helps you perform the task efficiently?

Yes, I am able to post jobs in a short time as the procedure is simple and meaningful.


4) Do you want to use this system in your daily life? Why?

Yes, this could help me a lot in hiring new employees in a short period of time.


5) What are the problems you find in the system?

The “Manage” could be done in a button and the fonts used in the interface could be enlarged so that all the users can perform the task more precisely.


Summary from interview:

In summary, the system is able to let the user post jobs easily. However, there are some improvements that can be done such as make the “Manage” in a button and enlarge the fonts used in the interface so that users can view all the info clearly and mitigate any mistakes that could be made.

User 4

Age: 26

Gender: Male

Education: Degree

Occupation: Programmer


Task 1


Observation:

When using our prototype, the user has no difficulty figuring ways to send a message to a job poster. The user completes the task in a minute by exploring the function of having a chat with the job poster via the interactions made available by our prototype. First, the user logs in and navigates to the main menu, where they can obtain details and lookup for or post jobs. The user afterward taps on the search jobs button to be taken to the jobs search segment. Following that, the user narrows the job search by looking for jobs using key phrases and the location of the jobs. The user then taps on the job post they desire to go to the job posting page, and then taps on the chat bubble button to go to the chat page and chat with the job poster. After the user obtains information he wants, he then goes back to the main menu by tapping the back icon and button. After going back to the main menu, the user then logs out the system.


Result from interview:

1) Is it easy to use the system to do the task?

Yes it is. It is very simple and the functions are easy to use.


2) What are the difficulties when doing the task using the system?

Hard to notice the chat bubble and the bubble didn’t indicate that it can let users have conversation with the job poster.


3) Do you think the system helps you perform the task efficiently?

Yes, as I completed the task in under a minute.


4) Do you want to use this system in your daily life? Why?

Yes, as it is quite easy to use. It may provide big help when I need a job.


5) What are the problems you find in the system?

The chat bubble is too small and users that use the system for the first time may not notice it and know that this bubble icon allows them to chat with the job poster.


Summary from interview:

Overall, the system can be utilised to flawlessly accomplish the duties of communicating the job poster; however, there is a lack of interaction design as the chat bubble icon is difficult to notice and users may not know that the icon provides the function.


Task 2


Observation:

Throughout our prototype, the user has no problems accessing methods for exploring ideal jobs to interview and apply for. The user completes the task in under a minute by accessing the function of uncovering job offers to interview for and applying for the job via the interactions offered by our prototype. The user first logs in and navigates to the menu page. The user afterward taps on the search jobs button to be taken to the jobs search segment. Following that, the user narrows the search process by searching for jobs using key phrases and the venue of the jobs. The user then taps on a job post related to the search result to go to the job posting page, and then taps on the apply now button to go to the application page. After that, the user applies for the desired job by filling up the application form and tap the submit application button to submit their application.


Result from interview:

1) Is it easy to use the system to do the task?

Yes it is. It is very simple and the functions are easy to use.


2) What are the difficulties when doing the task using the system?

I do not have any difficulties as every button is clear and easy to use.


3) Do you think the system helps you perform the task efficiently?

Yes, as I perform and complete the task in a short time.


4) Do you want to use this system in your daily life? Why?

Yes, as it is quite easy to use. It may provide big help when I need a job.


5) What are the problems you find in the system?

There is no notification that allows users to know whether the application has been accepted or not.


Summary from interview:

Overall, the system can be used to properly accomplish the duties of finding decent jobs to interview and apply. However the system can improve by providing users notifications when job posters accept or reject the application.

Task 3


Observation:

The user has no problems posting jobs. The user completes the task in under 1 minutes by achieving all of the criteria by utilising the interactions offered by our prototype. The user first logs in and navigates to the main menu. The user then taps on the post job button to post a job. After he taps the button, he will enter a page where it allows him to add a new job. Then the user taps the add a new job button and enters the job detail. Next the user taps the confirm button to post their job. After that the user will go back to the job post list page to manage their posted job. After tapping the manage button on the right of the post, they will be able to review the applicants by tapping the review applicant button. Applicant lists will appear afterwards and the user taps the manage button besides the applicant he wants to review. He accepted the applicant application by tapping the accept button. After that, the user goes back to the main menu and logs out the system using the log out button.


Result from interview:

1) Is it easy to use the system to do the task?

Yes it is. It is very simple and the functions are easy to use.


2) What are the difficulties when doing the task using the system?

I do not have any difficulties while completing the task.


3) Do you think the system helps you perform the task efficiently?

Yes, as I perform and complete the task in a short time.


4) Do you want to use this system in your daily life? Why?

Yes, as it is quite easy to use. It may provide big help when I become an employer.


5) What are the problems you find in the system?

The manage button does not have any button design, it may result in users not knowing that it was a functionable button.


Summary from interview:

Overall, the system can be used to properly accomplish the duties of posting jobs. However the system can improve by providing users with a manage button that has a button design.

User 5

Age: 25

Gender: Male

Education: Degree

Occupation: Occupational Therapist


Task 1


Observation:

The user has no trouble figuring out how to send a message to a job poster when using our prototype. The user completes the task in under a minute by navigating to the function of chatting with the job poster via the interactions provided by our prototype. The user first logs in and navigates to the main menu, where they can obtain information and look for or post jobs. The user then clicks the search jobs button to be taken to the jobs search section. Following that, the user narrows the job search by searching for jobs based on key phrases and location. The user then taps on the desired job post to navigate to the job posting page, and then taps on the chat bubble button to navigate to the chat page and chat with the job poster. After obtaining the desired information, the user returns to the main menu by tapping the back icon and button. After returning to the main menu, the user logs out of the system.


Result from interview:

1) Is it easy to use the system to do the task?

It is, indeed. It is extremely simple, and the functions are simple to use.


2) What are the difficulties when doing the task using the system?

There are no difficulties when completing the task.


3) Do you think the system helps you perform the task efficiently?

Yes, because I finished the task in under a minute.


4) Do you want to use this system in your daily life? Why?

Yes, because it is very simple to use. It could come in handy if I needed a job.


5) What are the problems you find in the system?

There is no major problem found in the system when doing the task as for me.


Summary from interview:

Overall, the system can be utilised to flawlessly accomplish the duties of communicating the job poster.


Task 2


Observation:

Throughout our prototype, the user has easy access to methods for researching ideal jobs to interview for and apply for. The user completes the task in under a minute by using the interactions provided by our prototype to access the function of uncovering job offers to interview for and apply for the job. The user logs in and then navigates to the menu page. The user then clicks the search jobs button to be taken to the jobs search section. The user then narrows the search by looking for jobs using key phrases and the location of the jobs. The user then taps on a job posting related to the search result to navigate to the job posting page, and then on the apply now button to navigate to the application page. The user then applies for the desired job by filling out the application form and clicking the submit application button.


Result from interview:

1) Is it easy to use the system to do the task?

It is, indeed. It is extremely simple, and the functions are simple to use.


2) What are the difficulties when doing the task using the system?

There are no difficulties when completing the task.


3) Do you think the system helps you perform the task efficiently?

Yes, because I finished the task in under a minute.


4) Do you want to use this system in your daily life? Why?

Yes, because it is very simple to use. It could come in handy if I needed a job.


5) What are the problems you find in the system?

There is no major problem found in the system when doing the task as for me.


Summary from interview:

Overall, the system can be used to properly carry out the duties of finding suitable jobs to interview and apply for.


Task 3


Observation:

The user has no difficulty posting jobs. By utilising the interactions provided by our prototype, the user completes the task in under 1 minute while meeting all of the criteria. The user logs in and then navigates to the main menu. The user then taps the post job button to submit the job. When he taps the button, he will be taken to a page where he can add a new job. The user then taps the add a new job button and fills out the job details. The user then presses the confirm button to post their job. The user will then return to the job post list page to manage their posted job. They will be able to review the applicants by tapping the review applicant button after tapping the manage button on the right of the post. Following that, applicant lists will be displayed, and the user should tap the manage button next to the applicant he wants to review. By tapping the accept button, he accepted the applicant's application. The user then returns to the main menu and logs out of the system using the log out button.


Result from interview:

1) Is it easy to use the system to do the task?

It is, indeed. It is extremely simple, and the functions are simple to use.


2) What are the difficulties when doing the task using the system?

There are no difficulties when completing the task.


3) Do you think the system helps you perform the task efficiently?

Yes, because I finished the task in under a minute.


4) Do you want to use this system in your daily life? Why?

Yes, because it is very simple to use. It could come in handy if I needed a job.


5) What are the problems you find in the system?

There is no major problem found in the system when doing the task as for me.


Summary from interview:

Overall, the system can be used to properly carry out the duties of job posting.

Observations

For Task 1, all users do not encounter any problems in figuring on how to send a message to a job poster when using our prototype. All users are able to complete the task within 1 minute. The only drawback can be identified when users are doing Task 1 is the issue with UI design of the chat bubble icon as the icon is difficult to notice at first place based on the observation and interview with all users.

In general, all users do not find any problems when doing Task 2 using our prototype. All users can complete the task under 1 minute by using the interactions provided by our prototype. However, we received feedback from User 4 regarding the improvement of the system by providing users notification when job posters accept or reject the application to improve the user experience when doing Task 2.

For Task 3, all users do not have major problems during completing the task using our prototype. All users can complete the task under 2 minutes by using the interactions provided by our prototype. However, there are several potential problems that have been identified by the users. One of them is the UI design for the ‘Manage’ function in managing job posts and applications that are small and not enclosed with borders which might cause confusion to users. Another problem is the lack of filters and sorting options when managing job applications. For example, rejected applicants are still visible on the list. The third problem is the lack of delete chat function, causing the chat list to be filled with old chats. The fourth problem is inconsistent words used such as ‘applier’ and ‘applicant’ are both used to describe the person who applied for the job. The last problem is inconsistent interactions provided by the system such as ‘manage’ text is provided for users to view applications by applicants whereas no button or text is provided for users to view chats with applicants as the system allows users to directly click on the chat bar with applicants to chat with them.

Findings


ree

 
 
 

Comments


©2021 by Gen Z. Proudly created with Wix.com

bottom of page