Hi everyone I hope you’re doing well and staying safe happy New Year, I hope you all enjoyed the New Year’s celebrations. I’m Trey from the UK I’m a totally blind electronic musician, Currently I make music with a combination of reaper hardware and modular synthesisers on Mac OS. Even though I use hardware modular synthesisers for my digital synthesis, I thought it would be cool for a long time if software modular could be made accessible to totally blind users. Recently I read about the new audulus4 update And it’s compatibility with the lua programming language. I know this is a longshot, and I have no coding experience myself. But does anybody think it would be possible to make audulus Accessible to blind users using the lua Programming language or other open source means? And would anybody be up for collaborating with me on this project? For an example of a synthesiser that is visual but that has achieved accessibility take a look at metosynth by Fuji juce: https://www.wooji-juice.com/products/mitosynth/
I’ve posted this here in the builds section because I didn’t know where else to post this I hope nobody minds. Thank you for taking the time to read this message, and I hope you can help me with this venture. I look forward to hearing from you all respect and blessings Trey.
Thank you for reaching out Trey! We’ve talked about how to make Audulus accessible to blind people, and it’s a really tough design issue.
For example, if you are wiring something, how do you give the user feedback that you’re approaching an input? How would you as a user know that it’s the right input you’re trying to go for if you have two identical nodes (or 10 or 50, as sometimes happens?).
I know you could possibly have metadata for screen readers on each I/O that would announce what it is when interacted with, but beyond that I’m not sure what else we could provide in terms of accessibility that wouldn’t fundamentally change what Audulus is in the first place.
As for Lua, the language allows you to draw shapes that the Audulus rendering engine itself uses to draw graphics, but it doesn’t just let you compile some other program that could adapt Audulus for blind people.
I’m saying all this not to discourage you, but just to say that we’ve given it thought, but have run into conceptual roadblocks. If you or others can help provide ideas to get around these roadblocks, we’re open to your ideas!
Thanks for this, I understand the problem you face. One simple way around this is to create a menu system that represents the available modules and inputs and outputs. The first part of the menu system Would basically be the module library. Then when the user selects the modules, The available inputs and outputs are displayed as menu options The user selects all the connections they want to make hits a button and the connections are automatically made for them. In terms of interface elements for example buttons knobs and sliders they just need to be correctly labelled so the screen reader can recognise them. What programming language is the UI written in? If you’re worried about the design being changed, You could create accessibility support as an add-on which users would download for free. Thank you for your reply and I would be more than happy to collaborate with you to make audulus accessibility a reality. I look forward to your reply kind regards.
Hi Trey, thanks for raising this. I’m also interest in accessibility (for me it’s a hands free perspective rather than vision but good to talk about).
I’ve been thinking about this for the last day or so and can imagine an interface for Audulus that might make this possible. If I understand correctly, blind uses generally interact with computers through the keyboard rather than a mouse (is that a reasonable assertion, Trey?) If so, imagine a workflow like this:
Press a hot key to enter module/node selection mode (either fully zoomed out or based on what’s visible).
Tab through all modules and nodes, with each announcing its title as you go.
Select the one you want with a key.
You can then tab through all of the elements of the selected module (potentially sub-setting by type e.g. press k for knobs, s for sliders, i for inputs, o for outputs, etc).
If you select a knob, arrow keys change the knob value.
If you select an output it then zooms back out for you to select a target module/node (again, tab through with announcement), select a module, tab through the available inputs and make a selection to confirm the connection.
Modules and nodes already have titles which is great, but there are of course some challenges:
What do you do with multiple elements with the same name? Perhaps some kind of “announce context” hot key would be helpful which searches up and down the graph announcing the other modules that it’s connected to.
It does rely on everything being well labeled and the community has generally gone towards not labeling most I/O at the module level in favor of more visual representations (that’s a community thing though not a technical barrier. The ability to label I/O is there and people creating for blind users could adopt other standards).
Deep patch editing with many of the same kind of node may be beyond blind users due to the difficulty of distinguishing between those nodes, but that shouldn’t stop performance patch interaction or high level patch creation from modules or basic patch editing.
A system like that which is fully keyboard based would also be useful for other accessibility needs, particularly hands free, because anything that can be done from the keyboard can be mapped to a voice command.
I appreciate there is a big gap between idea and implementation though
Hi sorry for my late reply things have been rather hectic for me recently just getting back to this now Your assertions regarding how blind people use computers are correct it is virus system of keyboard shortcuts and text to speech. Your concept for implementation seems pretty solid and I have a few additions. You posed the question of how blind people would interact with more complex patches, This could be achieved by having A hot key which speaks what are the connections are hotkey which would speak the currently selected connection. Regarding zooming back out to perform patch operations maybe they could be done through a series of context menus as opposed to zooming back out. A question you have to ask yourself is would you make this system self voicing using the computer text-to-speech engine, Or would you code everything up for compatibility with pre-existing screen readers VoiceOver on macOS and NVDA on windows. Another way to do this would be to have some kind of browser-based Interface which would be written in HTML5 would be textbased and have aria tags For the controls when you do things in the browser based interface the changes would take place within the software. You could have different browser headings for things like input connections output connections module controls a module selecter et cetera et cetera. You could make a piece of software which one activated by the blind user configures the add in for use with auduluss. No I’m not sure which method would be easier to implement from the developer perspective. It would be definitely interesting to get more peoples thoughts on this
