top of page

Audio Vision Reflection - Hugh F

Week 1-2

Opening Possibilities Rather Than Closing Things Down With Certainty

If there’s only one thing that I would take away from my whole time at RMIT, it is that investing time into rapid and imperfect ideation is an important key to obtaining a higher probability of quality work. The idea of initially being open to new ideas, no matter how ‘bad’ or ‘stupid’ they seem, can broaden the horizon of creative growth and invites constructive challenges to take place. 

 

There are many modes that the above philosophy can be abstracted into different frameworks; particularly into that of sound design and composition for visual material. A habit of mine – that constantly fails to follow this philosophy – has usually been to think of the first thing that comes to mind when designing audio in my assignments, and stick to it. When I do this, it is as if I walk hungry down the street and buy some porridge from a vending machine, only to find out that around the corner there is a really vibrant food court. That is, I pick one sound idea for a video that could work, and forgo an extra 10 minutes I would have needed in order to massage my subconscious into juicing 10 more ideas. Nine ideas could be terrible, but if just one out of the 10 is a cracker, then the end product of the work could be so much better than the first idea I had initially.

 

Healthy idea variation means minimising sound to the bare-bones. It also means heavily building a sound work, just to see what it would be like. My fault is that sometimes I feel too lazy or scared (both?) to step out of my comfort zone of musicality and venture into producing sounds I haven’t made before. While working on Gina Moore’s animation, I was intimidated by the erratic nature of the works because I thought music couldn’t work well with the movements of the animations. I had closed off any creative possibilities. “Ahh that won’t work, so I will not investigate further and just do something more comfortable”, was the first idea I had in my head. I did not budge out of habit. The flockingWhippets4 piece I did ended up being difficult to understand. My attempts at developing the right umwelt* that juxtaposed disgust and awe did not land with the audience, as I had not put enough time into idea cultivation to begin with.

 

*In the context of sound design, the concept of umwelt refers to the subjective and contextual nature of the sonic environment as perceived by an organism or entity, and it plays a crucial role in shaping the design and interpretation of sound in various media and environments. (Uexküll 1957)

 

In the context of sound design, the concept of umwelt refers to the subjective and contextual nature of the sonic environment as perceived by an organism or entity, and it plays a crucial role in shaping the design and interpretation of sound in various media and environments.

 

Reflecting more on my shortcomings, I think I just had the desire for the assignment to be ‘done’ or ‘ticked off my list’. This kind of thinking immediately bred a mindset of scarcity in the creative process. I went through Gina Moore’s selection of animations, picking the only one that jumped around the least – the whippets. It was the easiest for me to latch a premature idea onto without any other forethought about my choices. Instead of actually creatively exploring/expressing my unease with the erratic nature of the other animations, I went with the easy option. The worst thing is that I hardly even diverged my thinking from that point onwards. My thoughts went, “whippets–grotesque mice–I’ll make it sound gross–oh, this actually sounds ethereal–I’ll synthesise grotesque and ethereal and just hope people get it–submit” 

 

A Note On Relevance Realisation

 

After picking a Gina Moore animation in my comfort zone, without venturing as deep into my creative thoughts as I could have, I am curious to explore further in future projects. I look back on the Youtube lecture talking about relevance realisation, by  Dr. John Vervaeke. He talks about relevance realisation, and how it is involved in recognising patterns and connections between seemingly unrelated pieces of information (Vervaeke, 2020). I am looking forward to what other wonder and awe my subconscious could bring to my attention by investing in a deeper ideation process and using a broader range of stimuli and techniques to stimulate my creativity. By expanding my exploration beyond my comfort zone and delving deeper into my creative thoughts, I anticipate uncovering new insights and innovative ideas. Dr. John Vervaeke's insights on relevance realisation have inspired me to recognise the importance of identifying patterns and connections in my ideation process. Moving forward, I aim to embrace the challenge of exploring unfamiliar territories and experimenting with diverse approaches to nurture a richer and more impactful creative output. Through this journey, I hope to harness the full potential of my subconscious mind and unlock a wealth of untapped creativity.

Week 3

Diegetic Sound And Non-diegetic Music (Synthesised)

 

Gaining insights from Mark Hooper was a good way to start the week. He shared with us many things, such as technical knowledge, his career path, professional expectations of conduct, and self-care routines, to name a few topics. I learnt a lot of new things about being a sound designer.

 

One thing I did not know about was the sheer amount of sound data that someone like Mark has accumulated as a part of his arsenal. At his disposal he has over 8TB of sound effects that he can look up to use for employers and clients when he does sound for them. I appreciated the way he works, and just how dedicated he is to the craft. I fully understand and respect why he needs to store these sounds as he caters to content that requires the most realistic sound.

 

Under my own branch of taste, I am currently obsessed with getting away with a myriad of sound effects, just using a number of computer synths that I could count with my fingers. Of course, this sacrifices the realistic aspects of sounds that Mark would use. I like designing sound for media (mainly indie games) that can rely on the synthesised version of a dog’s bark, for instance. This does not mean that I would completely abandon the idea of recording and sourcing sound material from the physical world. I just enjoy the challenge of creating something meaningful, using my computer as a Swiss army knife for contexts that could thrive with my work style. I’m probably no Marvel candidate, and that’s ok with me. 

 

For me, crafting foley with synthesisers proves more challenging compared to making music, primarily due to the inherent nature of non-diegetic music. Non-diegetic music conveys its own meaning independently of the physical world or scene context. Foley creation with synthesisers demands a deeper comprehension of sound design and the skill to artificially recreate natural sounds. This skill is what I aspire to develop more as I step out of my comfort zones.

 

Delving deeper as I write this I think about sounds in the world surrounding me. The movement outside, my fingers on the keyboard, my fridge motor running, etc.. What is making me separate my idea of music from the sound effects I hear in my physical environment? Isn’t it all just vibrations? This is obviously the key to being able to design synthesised foley from a predominantly musical background; just try thinking of everything as music and go from there. 

 

I work with this spontaneous exercise on a rapid theoretical level...

 

The fridge is multiple waves of various frequencies and amplitudes humming at those frequencies indefinitely.

The vehicles on the road outside have a slight low hum, with soft high-frequency noise. Tie this in with a doppler effect (pitch automation) and a slight low-pass filter (frequency automation).

 

The keyboard would have various clicking at, say, 6/7kHz+ with some sort of pitch randomisation, accompanied by a synchronised, lower frequency hit to accentuate the thumping of my fingers on the keyboard.

 

All of these sounds would probably need completely different aspects to them as I am only theorising at this point. Theorising is none-the-less exciting.

Practicality

 

The main practical reason why I would prefer synthesised foley is due to the fact that, once implemented into a computer game, the game software would only need to tweak parameters of an in-built synthesiser, and not need to load a swath of mp3’s, thus making the game run at a more optimised level. This is done a lot with Web Audio API in online computer games that solely run using JavaScript.

 

 

Here are some researched examples of interactive media that use Web Audio API: 

 

A Dark Room: A Dark Room is a minimalist text-based game that uses ambient sound effects and background music to create atmosphere. The Web Audio API is employed to deliver these audio elements seamlessly during gameplay. https://adarkroom.doublespeakgames.com 

 

 

 

Gridland: A popular puzzle game developed by Doublespeak Games. Gridland is a unique blend of match-three puzzle mechanics and role-playing elements set in a charming pixel-art world. http://gridland.doublespeakgames.com  The videos for this have awful commentary so I did not add them.

Typatone: Typatone is an interactive web application that combines typing with music composition. It allows users to create music by typing words or sentences on their keyboard. As each key is pressed, Typatone generates a unique musical note corresponding to that letter, creating a melody based on the rhythm and sequence of the typed text. - https://typatone.com/ 

 

Patatap: A colourful and interactive website where pressing keys triggers different sounds and visual effects. It's a great demonstration of how the Web Audio API can be used for creating musical experiences. https://patatap.com/

 

This is a more user-friendly example of synthesised foley using DAW’s – a plugin called PANO Composer.

WEEK 4

Collaboration 

I started meeting up with Connie H from the animation class to discuss our work and the prospect of collaboration. Getting to know her a little bit was a good idea, as I was able to slightly gauge the aspirations she had that would eventually help form the demo I did for her animatics. Her references to music she wanted in the work fitted my style, and I was excited to put my own twist on it. 

 

Connie’s project design inquiry was as follows:

How can the struggles(efforts) of giving love be represented by cooking a variety of eggs in the form of 2D animation?

 

Sound references taken from Connie:

Music:

● Goofy, cute, lighthearted, and playful

● Reminiscent of the old 90s cartoon songs (but preferably without the lyrics)

● Can also include electronic sounds (bowowowow sounds y’know)

● Miu Mao

● Keeping My Head In The Sand

 

Sound effects:

● Stove Starting Sounds - (the beat will lead up to the start of the background music)

● Egg - Sizzling, Yolk pissing, Snoring

● Pan - The scraping of the spatula against the pan

 

A challenge I was coming up against after watching the animatic over a few times were things like getting the dynamic range correct at the right points. The storyline had multiple climactic moments, with a big one at the end, followed by a more sentimental, less goofy and lighthearted section. Synthesising and juxtaposing these feelings was going to be a learning curve for me, not only for the reason of knowing how to write the music, but also for being able to engage ideas mutually with Connie. 

 

We were off to a good start. I felt like we were both treading on eggshells so as not to hurt each other’s feelings about critical suggestions to the work. I was confident, however, that once we got to know each other, we would feel more comfortable giving and receiving constructive feedback.

 

I had shown Connie some recent work I had been doing, and how I would assign different musical melodies to characters. She liked this idea for her egg and girl characters, and encouraged me to give it a go for this project. We agreed to design some diegetic ‘crashes’ and ‘clicks’, and would try incorporating/synchronising them into the musical score as much as possible. 

I started just coming up with ideas based on Connie’s pitch presentation PDF. I came up with one demo while working with Connie in the animation classroom. I gave her a rhythmically swung and straight version of the demo, trying to align it with her referenced material as much as I could within the context of a demo.

 

 

 

 

 

 

 

 

 

 


 

 

At that point, I had received a more fleshed-out draft animation from Connie, and was prepared to venture further into dynamic contour in the coming days.

 

A note on sound frequencies and our association to light contrast.

 

In class we discussed why light and dark colours are predominantly associated with high and low frequencies respectively. I was particularly interested, as my weekly assignment had been on the black and white liquid video, Resonance - Momentary People, which in hindsight, I associated light to high frequencies and dark with low.

 

My own initial thoughts on the light-dark/high-low frequency association were as follows:

● Lower frequency sounds have longer wavelengths which means they can bend around obstacles and pass through gaps more effectively. The size of the obstacle relative to the wavelength determines how much diffraction bending occurs (Botteldooren et. al. 2009).  

A lot of time throughout human history, we have heard noises coming from behind obstacles that are visually unknown to us. This association of lower frequencies to darkness comes from our inability to perceive a sound’s source; i.e. the lower frequency sound is coming from the darkness or an unknown space. 

● If I think of an ant running around on the ground, I see it as a very light-weight creature. I also perceive it to have acutely quick (high frequency) movements as it scurries along. I use the same word ‘light’ with a different meaning of weight instead of colour contrast. But the association remains, and I can’t help but think of the word ‘light’ connecting metaphorically with higher frequency movements of sound waves. Author George Lakoff argues that metaphors are not just decorative elements of language but fundamental to how we conceptualise and reason about complex ideas (Lakoff & Johnson 1980).

WEEK 5

New Collaborator: Quenna W

One thing I took away from the A Bug's Life lesson in Audio-Vision class was the ability to open the story-perspective with a combination of sound design and a different feeling composition. 

 

Firstly, the exaggerated and chaotic sound effects accompanying Flik's actions immediately establish him as a clumsy and well-meaning character. These sounds provide exposition for Flik's nature, conveying to the audience that he is a bit of a bumbler, but ultimately kind-hearted and eager to help.

Secondly, the romantic soundtrack that accompanies Flik's attempt to assist the female ant suggests a potential romantic connection between them. The swelling music creates a sense of warmth and intimacy, hinting at the possibility of a deeper relationship developing between the two characters as the story progresses.

 

Another example that comes to mind for me is somewhat similar to that of what I mentioned about A Bugs Life. In the past week I picked up another animation-sound collaboration with Quenna W. Her piece follows two lovers who become separated, then reunited, by one of the lovers running across the ocean, fighting a bear, and breaking through walls. 

 

The animation’s exposition involves the two characters showing their love for one- another by doing all sorts of goofy things together, which is made apparent by the sound design mockups that Quenna presented. Because I had the romance covered in my music composition, it was then time to consult with my classmate and Quenna’s designated sound designer, Ryan. I talked with Ryan and Quenna about using sound design to encapsulate the goofy energy of the couple in love, while opting for a more sincere and caring sentiment in the music. I was then waiting on Ryan’s demo. 

Connie

As I continue to work by Connie’s side on her animation, we have run into more communication gaps. I remember being warned by Darrin that our collaborators would be potentially not as versed in sound and music meta-language. That is true in Connie’s case, and it’s a two-way street with my developing knowledge of animation lingo. Nevertheless, we managed to come to an agreement after some confusion.

 

At first Connie told me that she wanted to hear the characters ‘harmonise’ over the music more. Whether she was referring to the melody or the overall progression, it was hard to gauge. What I came away from the conversation with was that Connie was after something in the egg and girl characters that spoke to the underlying motif of the score, rather than a melody syncretic to the character movements as well as an underlying motif. I think she was trying to tell me that there was too much going on and that I needed to make it sound a little closer to a music video, and not as much of the One Man Band, Mary Poppins scene.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Connie also stated that she wanted some sound design done for the animation’s exposition. Taking what I learned from Don Hertzfeldt works I had watched, I was excited to try some sort of background home-ambience to accompany this exposition of the girl and the egg. I would try giving the exposition some real-life ambience in order to give the video a more open feeling, like ‘cracking open the oxygen’ and opening the audience’s imagination to the space the characters are in. To be continued...

Bibliography

W1-2

 

Uexküll, J. v. (1957). "A Stroll Through the Worlds of Animals and Men: A Picture Book of Invisible Worlds." In Schiller, C. H. (Ed.). Instinctive Behaviour: The Development of a Modern Concept. International Universities Press.

 

Vervaeke, J. (2020). "Ep. 33 - Awakening from the Meaning Crisis - The Spirituality of RR: Wonder/Awe/Mystery/Sacredness" . YouTube. Retrieved from https://youtu.be/_zkLevmQe90?si=fk3DzhQtkU6PzHgi&t=1191 .

W1-3

 

Doublespeak Games. (n.d.). "A Dark Room." In Doublespeak Games, Retrieved from https://adarkroom.doublespeakgames.com 

 

Doublespeak Games. (n.d.). "Gridland." In Doublespeak Games, Retrieved from http://gridland.doublespeakgames.com/ 

 

PANO Composer. (2020). PANO Composer, by Noise Makers. Retrieved from https://www.youtube.com/watch?v=f96XmN1JcQ8 

 

 

Patatap. (n.d.). "Patatap." In Patatap, Retrieved from https://patatap.com/ 

 

 

Typatone. (2015). "Typatone." In Typatone, Retrieved from https://typatone.com/ 

W4

Botteldooren, D., De Coensel, D., & Dekoninck, L. (2009). A Review of Sound Propagation in Narrow Streets. Building and Environment, Vol. 44, Issue 10, 1977-1987.

 

Lakoff, G., & Johnson, M. (1980). Metaphors We Live By. University of Chicago Press.

W5

Disney. n.d . Mary Poppins - One Man Band. YouTube. https://www.youtube.com/watch?v=6OwnBuPbMqc

Hertzfeldt, D. (2000). Rejected [Video]. YouTube. https://www.youtube.com/watch?v=MuOvqeABHvQ

W1-2
W3
W4
W5
Bibliography
bottom of page