Oculus Quest apps use AR to teach you how to play the piano

In a video posted to Reddit earlier this week, developer Dominik Hackl showed footage of his AR app, Magic Keys, running on Oculus Quest using the Experimental Passthrough API.

The footage, embedded below, was posted to Reddit and shows Hackl playing on a live keyboard while using an AR overlay on the passthrough view to learn songs in real time.

The concept is similar to VRtuos, which we talked about last year, and other similar VR piano apps available for Quest. The notes appear above the keys in sequential order, like a synthesis, allowing you to learn a song in real time without the need for existing scores or knowledge.

Piano apps on Quest had to use hand tracking and virtual piano calibration to align the virtual keys with your real piano. While it worked pretty well, there was still sometimes a little sense of disconnect between playing the real piano and seeing the virtual. However, using AR to present the superimposed notes on a live view of a real piano has the potential to be a game-changer.

What is particularly remarkable about Magic Keys on Quest is that it is an AR application, developed initially for tablets, then for HoloLens and Magic Leap One. Now that Quest’s experimental passthrough API is available, it has been ported to Oculus Quest.

Magic Keys is not yet publicly available, but Hackl says an APK will be released soon. However, there will be competition – existing Quest piano apps such as Great Reality and VRtuos have indicated that experimental passthrough support on Quest may be underway. In the latter’s case, a beta APK is already available for download on its Discord server.

Leave a Reply

Your email address will not be published. Required fields are marked *