Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I put sheet music into smart glasses [video]

youtube.com

Post reply on HN
Print view View original

21–30 of 72 posts

Re: I put sheet music into smart glasses [video]

#21

Post by yusina »

atoav wrote: post #15
vunderba wrote: post #10

Nice work. From the Github: > This allows the pianist to not have to turn pages, and more importantly, allows them to see the music and their hands at the same time, which is an unavoidable problem with traditional sheet music. I could definitely see this being beneficial for beginners. When I lived in a dormitory during uni I often played familiar pieces from memory pretty late on a digital piano (with headphones) i…

I wanted to say, after a certain level looking at your fingers when playing an instrument becomes the equivalent of looking at your legs when riding the bicycle. When I start to think too much about what my fingers are doing I will play worse. For if I want to practise a particular part where I get the fingering wrong, sure, but when you play it for real, looking is counterproductive. Something like this could be gre…

Counter point: people get too hung up on staring at the sheet. The sheet is just a tool to help you remember what you intend to play. The goal should always be to not need it anymore, and while using the sheet, it's like using a crutch.

The "looking at your fingers" challenge then becomes that you start to play "by eye" instead of "by ear" (or "by feel") which I find is very hard to overcome. Especially when you are improvising.

Though in a sense "by sheet" is just as bad.

Re: I put sheet music into smart glasses [video]

#22

Post by yusina »

rtrgrd wrote: post #11

Cool project! I have a suggestion - since the processing is done on a moderately powerful laptop anyway, is it possible to bypass the foot pedal and use audio (from laptop or glasses) to predict when to switch to the next bar? I assume it will be a complicated but would trying to match the FFT series to the sheet music pitch data work (or would harmonics cause major headcahes?)

Music transcription has been around for decades, algorithm-wise. In principle this one should actually be easier since it doesn't need to transcribe from scratch but "only" find where in a given sheet you are most likely to currently be.

(Next step: evaluate afterwords and point out mistakes)

Re: I put sheet music into smart glasses [video]

#24

Post by illwrks »

This is great!!!

A lot of digital pianos have midi out, (there was a midi recording tool posted here months ago by another HN member) I wonder if you could use that midi signal to keep what you see in sync with your playing to drive page turns? You could even add a karaoke like highlight to show the note being played.

Re: I put sheet music into smart glasses [video]

#25

Post by schobi »

This is really awesome!

I was surprised about using dilation. I would have expected music21 to support rendering to a certain resolution/dpi setting directly and avoid rescaling the images. But from the music21 documentation this is not obvious how to do it. Rendering music to a low dpi screen nicely (pixel perfect) could circumvent some of the hardware limitations in the mid term.

Re: I put sheet music into smart glasses [video]

#26

Post by gavinray »

I am so unbelievably excited for consumer-grade, useful AR.

There was a lot of hype around VR, but for the last 10 years I've been following progress on AR glasses.

The thing about AR is that it has the ability to enhance everything in your daily life, versus VR which is meant to be a separate experience.

Both Meta and Samsung are due to put out consumer AR glasses later this year and I think this might be the first wave of useful, daily-wear glasses we'll see.

Is there anyone who works in the AR space that could comment more?

Re: I put sheet music into smart glasses [video]

#27

Post by gavinray »

gavinray wrote: post #26

I am so unbelievably excited for consumer-grade, useful AR. There was a lot of hype around VR, but for the last 10 years I've been following progress on AR glasses. The thing about AR is that it has the ability to enhance everything in your daily life, versus VR which is meant to be a separate experience. Both Meta and Samsung are due to put out consumer AR glasses later this year and I think this might be the first…

Just bought a pair of the Mentra Mach1 glasses, let's see how they pan out.

Re: I put sheet music into smart glasses [video]

#28

Post by schwartzworld »

bambax wrote: post #17
vunderba wrote: post #10

Nice work. From the Github: > This allows the pianist to not have to turn pages, and more importantly, allows them to see the music and their hands at the same time, which is an unavoidable problem with traditional sheet music. I could definitely see this being beneficial for beginners. When I lived in a dormitory during uni I often played familiar pieces from memory pretty late on a digital piano (with headphones) i…

> I just stopped having to look down at the keyboard Maybe the next step is an app for people who don't read sheet music; it would light up the keys on the keyboard that you need to press, when you look at it...? Same for guitar, highlight where to put your fingers on a fret for each chord.

Various products have done this over the years. Forget app, actual keyboards and guitars that light up. It’s not a good way to learn.

Re: I put sheet music into smart glasses [video]

#29

Post by schwartzworld »

yusina wrote: post #21
atoav wrote: post #15

Earlier quoted context omitted.

I wanted to say, after a certain level looking at your fingers when playing an instrument becomes the equivalent of looking at your legs when riding the bicycle. When I start to think too much about what my fingers are doing I will play worse. For if I want to practise a particular part where I get the fingering wrong, sure, but when you play it for real, looking is counterproductive. Something like this could be gre…

Counter point: people get too hung up on staring at the sheet. The sheet is just a tool to help you remember what you intend to play. The goal should always be to not need it anymore, and while using the sheet, it's like using a crutch. The "looking at your fingers" challenge then becomes that you start to play "by eye" instead of "by ear" (or "by feel") which I find is very hard to overcome. Especially when you are…

> The goal should always be to not need it anymore, and while using the sheet, it's like using a crutch.

Uh, why? Lots of pros use sheet music, especially for complex pieces. I’ve never heard of an orchestra conductor insisting everyone be off-book.

It’s one thing to memorize pop songs or whatever, but nobody is out there shaming people for not memorizing Rachmaninov

Post reply on HN