The Augmented Spectacles Bringing Stories off the Page for Multi-Sensory Experience

The best reading experiences involve being able to immerse yourself fully, seeing the visuals play out in front of you while hearing the soundscape around you.

This year, Singapore’s National Library Board, in partnership with Snap and LePub Singapore, brought that sensation to life with the launch of Augmented Reading glasses.

Developed for Snap Spectacles, the reading experience is enhanced using text recognition and machine learning to trigger audio-visual AR effects in real-time. This includes soundscapes that use suspenseful music to set the scene, sound effects like doors creaking or distant chattering, plus visuals and effects that mirror the storyline.

To find out more, LBB’s Sunna Coleman speaks with Sergey Mast, creative director, code and art at LePub Singapore about designing tech that immerses rather than distracts, and how the product could evolve over time with advancements in ‘optical character recognition’ and generative AI.

LBB> Tell us about the brief from the client and the challenge they came to you with.

Sergey> National Library Board (NLB), Singapore has always championed lifelong reading. But with attention spans shrinking and competition from screens growing, getting younger audiences to pick up a book has become harder than ever. NLB didn’t want to create a marketing campaign. They wanted to reimagine what reading could feel like today, using technology, not as a solution, but as a spark to get Singaporeans back into reading.

LBB> How did you land on the idea of incorporating AR?

Sergey> Reading is naturally immersive, but today’s digital natives are used to richer, more interactive experiences. We thought: what if we could integrate digital elements that they are used to, without losing the essence of reading? AR felt like the right layer to quietly reawaken the imagination and enhance the act of reading just enough to help people fall into it again.

LBB> What was the biggest challenge in executing this project? How did you overcome this?

Sergey> Striking a balance between immersion and distraction. The temptation with AR is to go big in terms of visuals, animation and sound effects. But that contradicts the process of reading. We needed to feed the reader enough stimulation for their imagination to take over. This meant designing every visual and sound cue with restraint so that it complemented the book, not hijacked it.

The visuals pop up only when they help to enhance a scene. Soundscapes were subtle and ambient, used to build atmosphere. Gamification encouraged progress without overshadowing the process. We tested a lot of prototypes and fine-tuned until it felt seamless.

LBB> What did collaboration look like between LePub, Snap’s AR Studio, and the National Library Board? Were there any creative or cultural considerations unique to Singapore’s readership?

Sergey> This was truly a three-way collaboration. Snap helped shape the technical possibilities with their hardware. NLB brought deep insight into reading habits and genre preferences of the local target audience. And we worked to bridge the creative vision with a user experience that would feel natural and intuitive. Singapore has a highly digital-savvy population, which gave us the freedom to push the idea further. The challenge wasn’t accessibility, but making sure the AR felt intuitive and fluid.

LBB> What would you say to people who argue that AR risks replacing, rather than enhancing, traditional reading experiences?

Sergey> We’d say that we agree technology should never replace reading. Augmented Reading was built as a bridge, not a replacement. In fact, our hope is that people eventually feel confident enough to set the Spectacles aside and just keep reading. If this project gets someone to fall back in love with a book, then the technology has already done its job.

LBB> With technology continually evolving, how do you see this product improving and adapting over time?

Sergey> As with all technological advancements, more affordable hardware will make this more accessible. Advances in OCR (optical character recognition) and generative AI could allow for richer, real-time adaptations across any genre or language. But the core idea stays the same: tech that disappears as you read, helping your imagination do the heavy lifting.

LBB> How do you hope Augmented Reading Lenses will shape public perception of what libraries and books can provide in the digital age?

Sergey> We hope they challenge the idea that libraries are just places to borrow books. They can be incubators for digital creativity, immersive storytelling, and unexpected learning formats. Augmented Reading is a small glimpse of how analogue and digital worlds can coexist and even elevate each other.

LBB> What do you hope first-time users feel when they put on the Snap Spectacles and read a book?

Sergey> Curiosity. A little surprise. And most of all, that quiet joy of being pulled into a story again, maybe for the first time in a long time.

LBB> What feedback have you received from first-time users of Snap Spectacles?

Sergey> Apart from the initial surprise and curiosity, many people actually shared that it made them realise how much they missed reading, which was exactly the goal. Some users were skeptical at first, expecting something too gamified or distracting, but most were surprised by how natural and gentle the experience felt.

LBB> What kind of role do you think AR could play in education more broadly, beyond reading and storytelling, especially for visual learners?

Sergey> AR has huge potential, especially for learners who struggle with traditional formats. Whether it’s visualising complex science, stepping inside a historical moment, or reinforcing literacy with contextual cues, AR can make learning tactile and immediate. But it has to be done intentionally. The key is always to not replace the content, but to just support the process.