• The evaluation environment and participants/users are selected – these should both mimic their intended ‘real world’ context as closely as possible
• Depending on the project, participants are fitted with head or body mounted video recording devices; or devices are placed in/around the evaluation environment in such a way that they do not intrude or distract from the participant experience.
• Participants then experience the project within the evaluation environment, which is recorded using the equipment.
• After the participant experience, recording equipment is removed from the participant and video footage is transferred to a playback device (such as a laptop).
• The participant is then interviewed while the video footage of their experience is played back to them.
• This interview is also video recorded, ensuring that the participant and screen are both in frame as you’ll want to refer to what on-screen footage the participant is discussing.

Activity

Before you start

In our experience, video-cued recall is well suited to gathering participant data with projects that are somewhat technically resolved, such as a working prototype. We’ve founded that using video-cued recall with functional prototypes takes some of the pressure off worrying about technical issues that might arise when working with early or lo-fidelity prototypes and allows the design or research team to focus on managing logistics. Some key questions to address before implementing video-cued recall with your project include:

• How many cameras will you need to effectively record users experiencing your project?
• Where and how will these cameras be placed?
• Are there ambient light and sound issues to take into consideration?
• How will you efficiently transfer these video recordings to a playback device?
• When interviewing your participants, how will these interviews be recorded?

We suggest running a mock test of your intended data collection setup to help work through any potential logistical issues and make adjustments as required.

Activity steps

1. Setup your evaluation environment, prepare your interview questions and recruit your participants.
2. If relevant, brief your participants on what is expected of them and provide enough context to ensure that their experience of your project accurately reflects how they would experience it in a ‘real-world’ context of use.
3. Begin your evaluation. Depending on your project, this could take anywhere from 5-30 minutes. While participants are experiencing your project, you may take this opportunity to make your own observations and note these down for discussion in the following interview.
4. Transfer video of the participant/s to the playback device.
5. Conduct the interview and use your questions as a loose structure. These questions should address any key concerns, guided by your project aims, while also allowing the participant an opportunity to expand upon the discussion. Use the video of the participant experiencing the project to address specific usability and experiential elements of the project. Ask them what they were thinking and feeling at specific, relevant moments of their experience – this is where the video ‘cues’ their recall.
6. Ensure you keep copies of all video files – we suggest developing a rigorous archiving system as this will help you keep track of all your data.
7. Transcripts of these interviews, while valuable data, can be difficult to interpret without sufficient context. This is where the video recordings of the interviews become invaluable. Depending on how you intend to use your data, you might include additional context cues within transcripts or timecodes within transcripts to quickly navigate to specific sections of the interview video.

Notes

Students within the Creative Industries Faculty have access to the faculty Equipment Loan Centre (ELC), which is located on the ground floor of Z6 building at the Kelvin Grove campus. The ELC allow students to borrow out equipment to use for their studies, which includes DSLR cameras, video cameras, GoPro’s, tripods and more. If you intend to use video-cued recall to aid in your assessment work, discuss with your lecturer if borrowing from the ELC will suit your needs as this is a great resource available to you. Do keep in mind that equipment availability is subject to demand and loan periods vary depending on student and unit needs.

Creative industries equipment loan centre

• Location: Z6 Block, Level 1
• Email: ci-elc@qut.edu.au
• Phone: 07 3138 8121

References

1. Miller, Anne. 2004. ‘Video-Cued Recall: Its Use in a Work Domain Analysis’. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 48 (15): 1643–47. https://doi.org/10.1177/154193120404801503.

Octopus Clustering requires your ideas to be organised into individual, moveable elements – sticky notes tend to work best. It is best to ensure that when transferring your initial ideas to their moveable format that they are recorded in the same amount of detail, using the same pen or marker and onto paper or material of the same colour. The reasoning here is to ensure that each idea is given equal opportunity, as difference in detail, colour of pen and paper are all added variables – we want to focus on the ideas only and avoid all other potential biases and distractions.

Be clear about the aim of your clustering session – if working with a design team, what are you trying to achieve? While clustering is a great way to make sense of initial ideas, it is important that each member of the team is well informed about the project goal. A short discussion before commencing clustering, where the project aims are restated is worthwhile and a simple way to avoid potential issues around clarity. Clustering might also be used by a design team to address a specific problem, such as a usability issue or even to make sense of participant data. We encourage you to experiment with the method and adapt it to suit your projects on an as-needs basis.

If you’re clustering with users or participants as a way to involve them in the design process, ensure your notes are suitable to their demographics. For example, not using design or project-specific jargon if your participants are considered ‘lay persons’. Again, the rationale here is that this method can only be effective if your notes are clear, simple, legible and can be quickly interpreted by the people involved in the clustering activity.

Activity

Before you start

Firstly, you’ll need access to a space with both a large wall (around 2-3 metres wide) and plenty of room for your group to move around in. Randomly arrange your sticky notes into a ‘cloud’ across the wall and mark off the boundaries where the cloud finishes using tape, string or sticky notes of a different colour. Organise your group into 3-5 rows (at least 3 people per row) and brief them on the following:

• Row 1 are actively creating clusters and grouping the sticky notes.
• Row 2 are giving helpful suggestions to row 1.
• Row 3 are observing row 1, the emerging clusters, looking for any outlying or orphaned notes and shouting any useful suggestions about these to row 1.
• After 30 seconds has elapsed, row 1 will stop clustering, hand any notes they’re holding to the person behind them in row 2, and then step to the back of the row. In this process, row 1 becomes row 3, row 2 becomes row 1, row 3 becomes row 2 and so on. This cycling of people only happens in the form of entire rows – the rows do not mix between individuals.
• If your group is large enough to have 4-5 rows, rows 4 and 5 can discuss the emerging clusters with each other and prepare to offer advice when moving into row 3 and then row 2.

As a facilitator you have 3 verbal cues that will signal to your group what to do:

• “Empty hands!” – row 1 must stop clustering and hand their notes to the person behind in row 2.
• “Come out.” – row 1 will turn to the left and head to the back.
• “Step forward!” – row 2 step forward, becoming the new row 1 and begin clustering straight away.

Use a timer or stopwatch to keep track of time – aim for 30 second intervals of clustering with a few seconds of handover time.

Activity steps

1. Start clustering!
2. People might need to be reminded about what role is associated with their row, so don’t hesitate to remind them if you notice people are becoming lost or confused.
3. As clusters start to form over the cycles and you notice they’re remaining relatively unchanged there might be outlier or orphaned notes – point these out to the last row if they’re not noticing them.
4. Aim for around 5-8 clustering cycles – you may need more or less to arrive on solid clusters. This is up to you, the facilitator to determine based on your observation.
5. Give the final row a warning that they are last and once their 30 seconds has elapsed, cheer and applaud the group for their work.
6. As a group, encourage everyone to step back and get an overview of the final clusters – do any need to be merged or is everyone happy with the outcome? Each cluster should be labelled with a heading and a short phrase or sentence to describe it.

References

1. Stickdorn, Marc. 2018a. This Is Service Design Doing. 1st edition. O’Reilly Media, Inc. QUT Library Permalink.
2. Stickdorn, Marc. 2018b. This Is Service Design Methods. 1st edition. O’Reilly Media, Inc. QUT Library Permalink.

The rich data produced from cultural probes is difficult to analyse or produce any definitive conclusions from beyond themes or groupings of related findings, meaning the method should not relied on as the sole basis of producing findings for a study. This being said, data produced from cultural probes is often valuable in informing and inspiring new design directions for a project, as participants are encouraged to reflect on aspects of their lives that could be easily overlooked by observing them or them not willing to share overly personal information when being interviewed. Herein lies the value of cultural probes, as when they’re well designed, they enable participants to share latent information that may seem mundane or inconsequential to them but may be invaluable to a researcher or designer.

While the overarching aim of cultural probes is to facilitate divergent ideas and exploration, tailoring kits to suit a specific context (i.e. interaction design) can be especially useful for emerging designers as the findings will often be easier to organise. Another key aspect to keep in mind when designing your kit of probes is that the tasks you expect to participants to complete must be engaging! A key issue with this type of method is the expectation that someone will simply engage with the probes because you have asked them to. It is up to you, the designer, to carefully craft probes that engage your participants as this engagement is the ‘reward’ for participating in your research.

Activity

Before you start

When developing a cultural probes kit there are a few important questions to address, as responding to these will inform and shape the kinds of probes you include. These questions include:

• What is the primary research focus?
• How do the probes address the research?
• What kind of data will the probes offer you?
• What are the time constraints?
• How will you gather your participants?
• How will you ensure participant engagement?
• How will you deliver the kits, and ensure they are returned?
• What incentives, if any, will you offer participants in exchange for their involvement?

As you can see from this list of questions, carefully crafting and executing cultural probes successful takes time and be quite a laborious task. Don’t let this discourage you from exploring cultural probes in your projects but do keep the logistical difficulties in mind when deciding whether or not this method is suited to your project constraints.

While it might seem impractical to consider using cultural probes as part of an assessment item, given the short time frame, we have created an activity that focuses more on the process of implementing probes that are accessible and allow for a relatively short turnaround time. The activity below focuses less on the design of context-specific probes and more on gathering general insights into the day-to-day lives of prospective users, which are recorded digitally and appropriate technologies they already have access to.

Activity steps

1. Select your participants. A group of 3-4 people is well suited to a project with a limited timeframe as this will enable you to deploy, track and collect responses and should leave enough time to integrate your findings into further design development.
2. Prepare your kit of probes. Firstly, choose a digital platform that enables image and text uploads. Blogging platforms such as WordPress, Blogger and Tumblr are all accessible and offer the basic upload functionality required, although there are plenty of alternatives. Consider working with your participants to determine the sharing platform that suits them best is another option, which also may reduce the learning curve and time commitment expected from them.
3. Use images, text and diagrams to explain tasks. Open ended questions allow participants to provide more detail about their experiences. Set a small series of tasks that each participant can complete and document via your chosen sharing platform. Images can be documented and uploaded using a smartphone – the key here is to encourage participants to work with the technologies they’re familiar with to reduce their perceived effort.
4. Brief the participants on the expected time commitments and process.
5. Deploy the probes – about 1 week with daily documentation from each participant is a good amount of engagement to aim for.
6. You can keep track of participant engagement by checking in on their blogs/wherever the kits are hosted. Check in with participants to see if they have any questions about tasks.
7. Once the time period for your probes has elapsed, collate the data and consider using an organisational method to interpret and analyse it such as Octopus Clustering.

References

1. Gaver, Bill, Tony Dunne, and Elena Pacenti. 1999. ‘Design: Cultural Probes’. Interactions 6 (1): 21–29. https://doi.org/10.1145/291224.291235.

Thales Macedo from Hyper Island¹ is credited with developing this method and we encourage you to build on the steps described below and in the activity as the beauty of this kind of method is in its flexibility and adaptability. Much like the PACT framework and SCAMPER method, Hot Air Balloon provides a series of organising principles that can be used to break down a problem, providing new insights into the problem and often uncovering aspects that might be easily overlooked when addressing the ‘big picture’ of them problem at hand.

While this method follows a relatively common approach to problem-solving – structuring and organising a problem into a series of categories or themes, its innovation lies in its use of metaphor as a way to enable people of varying technical ability, education level, lay persons and designers to contribute relatively equally. The use of metaphor encourages creative thinking regardless of skill and experience level, while the collaborative nature of the method is well suited to both interaction designers and team-based projects from any domain.

If you’re using Hot Air Balloon for the first time, we encourage you to include each of the following seven steps – do feel free to adapt, remix and expand on these steps on an as-needs basis, as a project may have its own unique requirements. Finally, while the method is not context-specific to interaction design, we have included some prompts below to help contextualise each step to interaction design, which are based on how we’ve used the method in our own projects – with great success too!

The Hot Air Balloon steps are:

• Wind: External forces we have no control over such as time, budget, material and equipment constraints – this also might include the experience and technical proficiency of your intended users;
• Sandbags: Internal challenges you and your team are facing, such as technical ability, time management and communication;
• Hot Air: Your team and project strengths – skills, abilities, personal qualities, innovation in the project, a desire to evoke change;
• Passengers: These are the internal stakeholders that have influence on the project –the immediate project team.
• Observers: The target demographic/s for the project, as well as external funding bodies, community groups and any organisations connected to the project beyond the immediate team.
• Paradise Island: This is the big picture goal – envision the intended, lasting impact your project might have in 5-10 years – this is your paradise! How do you imagine the future will be changed or influenced by your project?
• Flight Path: Your flight path is determined by the decisions made that influence the direction of your project. What are the actionable steps that can be taken today to steer your balloon towards paradise island?

Activity

Activity steps

1. Write down each of the 7 headings across a whiteboard or large sheet of paper.
2. The facilitator should guide the team members through each heading, step-by-step and reiterate what the headings mean when moving on to the next. Spend about 5-7 minutes on each step.
3. Team members should share their thoughts by writing under the heading or adding a post-it note below.
4. When a team member contributes, they should briefly describe their contribution. It’s up to the facilitator to keep discussion brief and constructive.
5. The facilitator should encourage input from everyone in the team – the aim here is to create shared ownership of the project scope and vision.
6. The final step – the Flight Path – is where the team develops an action plan to put into practice. Each team member should agree on this action plan, which can be drawn up as a separate document and distributed across the team.

References

1. Macedo, Thales. n.d. ‘Hot Air Balloon’. Hyper Island Toolbox. Accessed 26 February 2020. https://toolbox.hyperisland.com/hot-air-balloon.