RIT will permanently shut down the RIT Wiki (Confluence wiki) on February 2, 2024. Review the Wiki Shut Down Overview for information on alternative tools, exporting content from the wiki, and other FAQs.
Team Vision for Problem Definition Phase
Plan:
- Understand the scope of the project overview
- Understand the current and desired state of the project
- Interview customer
- Draft Use Cases, Customer Requirements, Engineering Requirements
- Define team roles and norms/values
- Begin populating Confluence page
Actual:
- Gained a general understanding of the current state of the project
- Conducted preliminary interview and defined customer requirements and deliverables
- Assigned team roles and formed a group of norms and values to follow
- Filled out required Confluence pages
Project Summary
The automated player piano is a continuation of a multi-year effort to retrofit an existing piano with a device which will allow the piano to be played completely autonomously or as a compliment to a human pianist. Ultimately, it could be used by people who have suffered hand injuries (re)learn how to play the piano and as a practicing tool for people to learn new techniques. The piano's functionality must not be hindered such that the piano could still be played without interference from the device. Currently, there exists a device that is capable of playing the piano however this device is in a state of disrepair due to limitations caused by the COVID-19 pandemic.
The goals for this project are to realize the scope of the previous team's work, complete unfinished features, and continue with new features determined by the stakeholders. The aforementioned features that are set as the goals for production are to complete the unfinished construction of the device, develop a system to introduce a new layer of dynamics by automating the sustain pedal, and to continue development on a feedback system. The constraints that will effect the development of these desired features include working within the bounds of the previous team's scope, ensuring that the piano itself is not damaged or augmented in a way that normal operation will be hindered, enabling different modes of play (full autonomous, half autonomy-half player, and full player control), and the final constraint is to have a fully operational system of song selection and playback.
Use Cases
These are the primary use cases planned for the Piano Player. These flowcharts are designed to highlight potential problem areas in the use of our device.
Student/Teacher Case
- This use case considers the scenario of a Piano Teacher Providing a Student with complete or supplemental lessons/exercises. These would likely be performed by the piano and include the student playing along with the MIDI performance.
Amputee Use Case
- This use case considers the scenario of an amputee wanting to practice playing the piano. Whichever appendage is missing, there will be MIDI files to choose from that can play that part.
Student Self Practice Use Case
- This use case considers the scenario of a student who wishes to practice a piece.
Physical Therapy Use Case
- The use case explores the potential of using the piano player device as a physical therapy device within a hospital setting. Automated accompaniment can help facilitate piano play for any tempo/skill level.
Duet Use Case
- This use case considers members of a duet who are unable to rehearse together ahead of a performance. One member of the duet (Pianist 1) will practice and record themselves playing, then send the recording to the other member (Pianist 2). Pianist 2 can then play along with the recording, practice, and return a recording to Pianist 1. This process can be further iterated if needed.
Project Goals and Key Deliverables
An intuitive automated player piano system which allows players to play with full control, one hand/duet modes, and fully autonomously. Previous work done by past teams will continue to be developed and improved on. Some major deliverables include:
- Put together the work from the previous team which got interrupted by COVID-19 pandemic
- Working feedback system that can tell the player what keys were pressed
- A focus on increasing the usability of the device
- All 88 keys functioning as well as the sustain pedal for added dynamics
- Song lead-in/metronome to get ready, scrub feature to play a specific part of any song
Customer Interview
Current State:
- What components of the current device are in working condition?
- Last he saw the piano, all of the keys had been put back in. Components designed by the previous team have not been integrated into the piano. Upper and lower keys needed to be removed? All parts should be present. Assemble the components. Last year the range/dynamics were a focus. Volume of key press dictated by the speed of the key press. High requirement is dynamics
- What current aspects of the piano do not meet your expectations?
- Feedback system at the beginning of the year last year. Whatever you were playing would be compared with the actual song. Record what you played, playback?, Do some sort of comparison, diagnostics or something. Buttons and microphone mounting. Volume from microphone provides the dynamics, buttons do not have a volume. Microphone also does not have the ability to distinguish notes individually. Google API package, upload a sound file and it spits out a MIDI file. MAGENTA?
- How was the sustain pedal previously automated?
- A couple of years ago there was a linear actuator. MIDI files do have a sustain pedal. Never implemented in software though. Was not a requirement that year. There was a sustain and other pedals actually, damper and low octave sustain. Nice to have. Expected there to be a sustain this year.
Desired State:
- Is there already an accepted audience that is waiting for this product?
Side project, we could start looking for a potential home for this product
Are there features that are not yet implemented that you would like to see added to the piano?
Dynamic control, playback, really all the requirements have been defined already. The only must-have is full packaging (mounting/where to put UI or phone), Power supply/plugs, most of the functionality already covered. See the full list of what we are considering requirements so then he could potentially add more
Where do you envision this project to be in April of 2021?
Expect feedback system in place. Given that it will play, with dynamics, features such as record and playback, scrub feature to play a specific place of the song, sustain pedal, seeing where in the music you are. Lead ins for music, a few empty bars to get ready. Start recording at some point. ***Extra focus on usability***
Key Goals/Deliverables:
The project has gone through several iterations; is there a timetable or definition to a minimally viable product?
See previous question for most of the full project. Lead in mandatory this year, see the music is a Nice to Have. Like the thing to be usable. This is not a mass production product, custom design but it must be robust. Stability, if the piano is moved does stuff shift? Calibration mode stays.
If we were to deliver a user manual with the finished product, what form would you like to take?
Yes, good thought. We could discuss this more. Point of need help bubbles? Read-me-first? Need to know the customer and who the customer is before setting requirements around it. Formatting easy, content is the harder part. Start documenting these things. High priority. May not know how to open PDFs.
From the previous team's unfinished work, what would be the priority to complete?
Priority to complete unfinished work is to assemble everything from the previous semester.
What is the most essential aspect of this device?
To assist somebody playing. It's not a focus on having a piano that plays itself. Have the piano SUPPORT the player
What is the first location you would place the device? Where do you ultimately see it being used?
Would require more investigation, rehab facility amongst physical and mental rehabilities. VA hospital or rehabilitation center. Children’s hospital. School environment.
Key Constraints:
Are there any weight, material requirements for the project?
Not weight wise, ability to be able to be plugged into house power, already somewhat mobile (can it get in and out of rooms or elevators). Has to be safe, temperature is a big engineering requirement, do not get so hot that it becomes a serious problem
What is the budget for our project?
I don’t know. He's only the customer for a department-sponsored project. Last year it was pretty small. Question for guide or Dr. Debartolo
After several years and iterations of the piano what pitfalls have you seen previous teams fall into that you would advise against?
Keep lines of communication open between each other. For the most part they were good teams, but sometimes some members weren’t pulling equal weight. Be very clear, set time based deliverables. Plan in backfilling just in case somebody is unable to do their work. Test everything
Customer Requirements (Needs)
Document Owner: Nick Besley
Created 09/01/2020
| Customer Rqmt. # | Importance | Description | Comments/Status |
|---|---|---|---|
| 1 | High | Must play any standard MIDI arrangement completely autonomously | Only require a max of 10 keys at one time, must be able to select either hand as well |
| 2 | High | The piano plays the notes typically played by one hand while the user plays the notes of the other hand. | 9/23/20 - Customer added this as a high priority via feedback |
| 3 | High | Piano still can be played normally, with no obstruction, all panels close fully | |
| 4 | High | Ability to record performance as MIDI file | |
| 5 | Med | Piano cannot be permanently altered | 09/09/2020 - Customer indicated this was not a high priority anymore and can be relaxed |
| 6 | High | Components must not pose any safety risk | |
| 7 | Med | Should sound as much like a human player as possible | |
| 8 | Med | Should be completely stable/stationary within the piano | |
| 9 | Med | Device supports sustain pedal | |
| 10 | Med | Include support for a count-in with a metronome. | |
| 11 | Med | Piano can play full 88-key range | |
| 12 | Med | Create a user manual | |
| 13 | Med | Improve/consolidate all electronic/mechanical systems | |
| 14 | Low | Volume/tempo controls within UI | |
| 15 | Low | Master shutoff switch/volume | |
| 16 | Low | Improve playing/training/practice interface |
Engineering Requirements (Metrics & Specifications)
Document Owner: Nick Besley
Created 09/01/2020
| Rqmt. # | Importance | Source | Function | Engr. Requirement (metric) | Unit of Measure | Marginal Value | Ideal Value | Actual Value | Test Result | Direction of Improvement | Comments/Status |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | CR 4 | Feedback system function | Time from key press to recorded feedback | Seconds | 0.25 | 0 | - | - | - | A hardware-based feedback system allows for manual playing to be recorded | |
| 2 | CR 14, 15, 16 | Intuitive UI | Satisfaction | User Feedback Scale of 1-5 | 3 | 5 | - | - | - | User should be able to easily control volume and tempo as well as the mode that the device is operating in through the use of simple UI | |
| 3 | CR 7 | Mechanical noise from solenoids | How loud is the device? | Decibels | 15 | 0 | - | - | - | The device should be as silent as possible; 9/23/20 - ideal value updated per customer feedback | |
| 4 | CR 10 | Lead in/metronome | Time between 'Play' and part of song selected | Seconds | 5 | 20 | - | - | - | Does the user have enough time between setting up the song and playing, a lead in | |
| 5 | CR 5 | Visual Defects with piano | Our work does not hurt the piano | Number | 1 | 0 | - | - | - | Device cannot damage/hinder play-ability of the piano | |
| 6 | CR 1,2, 7 | Ability to play any song like a human and autonomously | Time between key presses | Seconds | 0.01 | 0.03125 | - | - | - | A key must be able to be play 16th notes at 120 bpm | |
| 7 | CR 12 | User Manual | Satisfaction | User Feedback Scale 1-5 | 3 | 5 | - | - | - | A user manual should exist to allow people to learn how to operate the device | |
| 8 | CR 2, 16 | One Hand Operable | Satisfaction | User Feedback Scale 1-5 | 3 | 5 | - | - | - | The player should be able to interact with all buttons and interfaces with only one hand. | |
| 9 | CR 1 | MIDI Compatibility | MIDI Conversion | Key Played/Notes in MIDI | 10 | 88 | - | - | - | ||
| 10 | CR 6 | Safety | Follows Health and Safety Protocol | Passes EHS or related health entity review | Pass/Fail | Pass | - | - | - | This engineering requirement is not steeped in actual numerical composition; the product simply must pass safety protocol and review by a proper entity as we follow best practices | |
| 11 | CR 8 | Device Stability | How far can it move once installed | Millimeters | 10 | 0 | - | - | - | Does the device stay calibrated after the piano is moved around/over time as the piano is operated 9/23/20 - scope updated per customer feedback | |
| 12 | CR 11 | All 88 Keys are usable | Number of keys playable | Number | 86 | 88 | - | - | - | ||
| 13 | CR 9 | Sustain Pedal Support | Time from pedal press, to feedback signal reaching computer | Seconds | 0.1 | 0 | - | - | - | We want the pedal press to register in the computer in real time, and minimize how long it takes. | |
| 14 | CR 9 | Sustain Pedal Support | Time from signal sent to pedal, to pedal being pressed | Seconds | 0.1 | 0 | - | - | - | We want the computer to be able to press the sustain pedal actuator in real time and minimize how long it takes | |
| 15 | CR 7 | Key Volume/Dynamics | Velocity of solenoid release → velocity of key press → dynamic of note played | Decibels | 40±5→100±5 | 40→100 | - | - | - | 9/23/20 - Customer commented this as an additional requirement to cover in terms of piano paly | |
| 16 | CR 6 | Power Usage | Total Amps used by system (assuming 15A breaker) | Amps | 10 | 5 | - | - | - | 9/23/20 - Customer wanted this base covered as to not pose any power regulation errors |
Benchmarking
Benchmarking: Market Products (Steven Petrick)
Brand/Device: | PianoDisc Prodigy | Yamaha Disklavier | QRS PNOmation |
|---|---|---|---|
Automation Design Style (Electronic/Analog, etc.) | Electro-Mechanical with App Integration | Electro-Mechanical USB/LAN to Device or App | Electro-Mechanical with App Integration |
| Key-press Mechanism | Solenoid Array | DSP Servo Solenoid System | Teflon impregnated solenoid Array |
| Playback Library | Pre-loaded Music and Custom MIDI Files | Pre-loaded Music and Custom Files | Pre-loaded Music |
| Feedback System Style | Optical Sensors into MIDI (ProRecord) | Optical Sensors into MIDI, .wav, .mp3 | Optical Sensors into MIDI (PNOscan) |
| Piano Integration | Attached device with system hidden inside piano | Custom Piano | Attached system Hidden inside Piano |
| Controls (Volume, Tempo, etc.) | In-app | On Switch Box | In-app |
| Silent Mode | Yes (QuietTime) | Yes | Yes |
Benchmarking: Apps for Learning Piano (Jeffrey Seamon)
| App: | Piano Academy - Learn Piano | flowkey: Learn Piano | Simply Piano by JoyTunes |
|---|---|---|---|
| Developer | Yokee | flowkey | JoyTunes |
| Price | $0 (w/ In App Purchases) | $0 (w/ In App Purchases) | $0 (w/ In App Purchases) |
| On Screen Display | Keyboard, scrolling sheet music | Keyboard, scrolling sheet music | Keyboard, scrolling sheet music |
| Instruction | Tutorial videos from personal instructor, training games | Interactive step-by-step courses, video tutorials, ability to slow down songs | Step-by-step courses, ability to slow down songs |
| MIDI Connection | Yes | Yes | Yes |
| Feedback | Instant | Instant | Instant |
| Songs | Classical to contemporary | Classical to contemporary | Classical to contemporary |
| Audience | All ages/levels | All ages/levels | All ages/levels |
| Languages | 19 | 12 | 11 |
Benchmarking: Previous Senior Design Teams (David Anthony)
| Team: (title links to page) | P13363-Touch Pad Piano Keys | P13364-Piano Touch Keys II | P19002-Project Grip |
|---|---|---|---|
| Year | 2012-13 | 2012-13 | 2018-19 |
| Objective | Controlling parameters of music with one hand while playing with other | Controlling features such as pitch without compromising playing hands | Gamify prosthetic use for independent rehabilitation |
| Technology | Trackball Built into Keys | Capacitive Touch Sense | Power Glove: Flex Sensor, Accelerometer |
| Dimensional Control | 2 Dimensions (X and Y Axis) | 2 Dimensions (X and Y Axis) | At least 2 Dimensions (Finger Flex, Rotation) |
| Responsiveness | Real Time | Real Time | Real Time |
| MIDI Compatibility | Yes | Yes | N/A |
| Runnable on Existing Hardware | Yes, w/ modifications | Yes, w/ modifications | Yes |
Benchmarking: Different Player Pianos (Nick Besley)
| Device: | Antique Player Pianos | QRS Music Technologies Pianomation OT | PianoDisc |
|---|---|---|---|
| Year Developed | 1920s | 2018 | 1988 |
| Feedback System | None | Unable to play manually with the device installed, no feedback system | There exists an instantaneous feedback system capable of playing performances back |
| Song Format | Paper scrolls with perforations that correspond to each of the 88 keys | 5 pin MIDI, Bluetooth MIDI, USB MIDI inputs | MIDI compatible, "iQ Player App" used to select songs |
| Power Requirements | None, the entire system was mechanical. Powered by foot pedals | Standard wall plug | |
| Tempo Ranges | Limited by a switch on the panel to common tempos at the time | ||
| Volume Control | Depends on how hard you pedal | ||
| Method of Playing | Full autonomous, full manual control, unsure if single hand scrolls were sold but it would be possible to play one handed or to duet. | Full autonomous, can not play manually without removing the device from the piano entirely. | Full autonomous and full manual control |
| Automated Key Pressing | Pneumatic, compressed air and a series of valves. All 88 keys, no sustain pedal | Electro-mechanical solenoids, no sustain pedal by default. Is available as an add on | Electro-mechanical solenoids installed underneath the keys of the piano (Grand or Upright). No sustain pedal |
| Notes | https://en.wikipedia.org/wiki/Player_piano | https://cdn.newswire.com/files/x/77/43/ | https://prodigy.pianodisc.com/what-is- pianodisc/ |
Benchmarking: Variety of "Keyboard" Instruments (Josiah)
Criteria | Upright Piano | Harpsichord | Rhodes |
|---|---|---|---|
Type of Actuator | Hammer on string | Plectrum across string | Hammer on tuning fork/tine |
Volume Controls | Determined by press velocity | None | Amplifier |
Power Required | No, acoustic | No, acoustic | Electric Amplifier, similar to electric guitar |
Range | 7.5 octaves | 4-5 octaves | 6-7.5 octaves |
Sustain Capabilities | Sustain pedal | No sustain aside from hold down key | Optional sustain pedal |
Tone | Dynamic, reliant on player character to change sound, balanced harmonics | Generally very harsh, monotone, lots of high freq harmonics | Warm, buttery sound, pure tones are very strong |
Number of simultaneous voices | Number of keys | Number of keys | Number of keys, but can be limited by pickup/output type |
Constraints
- Device must be easily used by persons undergoing physical or mental rehabilitation
- Device must use standard wall power
- Piano must not be permanently modified or damaged
- Device must be stable and fit under the piano cover
- Unknown budget
House of Quality
Document Owner: Nick Besley
Date: 09/05/2020
Updated: 09/25/2020
You can find the working document at the following link: House of Quality Google Docs
Problem Definition Review - 9/9/2020
See the Problem Definition Review Notes sub-page for more information
The following link leads to the pre-read of the review: Problem Definition Review Pre-read. A PDF version is available here.
The following link leads to the presentation given during the Problem Definition Review: Problem Definition Review Presentation. A PDF version is available here.
Plans for Next Phase
Team Plans for Next Phase:
The next stage, or sprint, contains the System Level Design portion of the project. The design of the project has already been fleshed out, so the primary objective is to both complete requisite documentation and review prior design and implementation. The first portion of the sprint will last about a week and a half and contain organizational matters such as Benchmarking and Concept Development. The second phase, set to overlap with the documentation portion, will be the team's familiarization with the core systems design, product implementation, and discipline orientated understanding of the piano.
Individual Three-Week Plans:
Jeffrey Seamon End of PDR Phase Three-Week Plan.pdf
Nick Besley End of PDR Phase Three-Week Plan.pdf
Steven Petrick End of PDR Phase Three-Week Plan.pdf
David Anthony End of PDR Phase Three-Week Plan.pdf
Josiah Martuscello End of PDR Phase Three-Week Plan.pdf