In the early days of Synapse, we had a client who would come to us 6 weeks before an industry trade show every year with a list of product concepts that they needed working prototypes of. We would spin up a team of engineers and make 6-8 prototypes in a few short weeks. The team motto was “Done is Beautiful”*. We weren’t constrained by product cost or manufacturability, so every decision we made was first and foremost about what would get us working parts on a near-impossible schedule. Since then, Synapse has worked on a lot of products that go into mass production, and while done is still beautiful, it’s more difficult to define. You never have the time or budget to reach perfection before shipping, so how do you decide how good is good enough?
You never have the time or budget to reach perfection before shipping, so how do you decide how good is good enough?
With software, you can usually make updates after you’ve released your work into the wild. Hardware is much more difficult to upgrade. Consumer products get used for years, and industrial products can be in service for decades. Releasing hardware that’s not mature enough can be bad for your brand and cost you lots of money in customer service and product returns.
Before we get into the details of how to tell if you’re done, we want to remind you that the cost and complexity of making changes to a design rises substantially as you get closer to manufacturing. This blog post isn’t a checklist of items that you should consider the month before you’re supposed to ship; it’s a set of things you should consider throughout your development process. Spending time up front to understand your users, define exactly what your product is and isn’t, and prototype areas of design risk is always a good investment.
Have you tested prototypes with real users?
The best way to preview what will happen once you launch is to get real users using your product. Give a prototype to one of your potential customers and try not to hover or explain too much. What users initially say is often telling about what design features they will gravitate towards in the end. They might not even notice the things you’ve decided are most important. Every feature you have in a product takes time to specify, design, and test and a great way to ship products faster is to only design in the features that have sufficient value to your users. User testing can also save you valuable time in re-design of features after they’re implemented. For example, one product we built had a “trigger” button that was so seamless in the industrial design, users often had to be taught where the button was. A slightly larger reveal space between the parts solved the problem and the button became intuitive.
Leaving prototypes with users for days or weeks can tell you a lot about product robustness. You know how expensive your pre-production units were to build, and you treat them gently. Real users are not gentle. On many products we’ve worked on, particularly wearables, simulation and fixtured cycle tests rarely replicate the failure modes we see in real-world use. If your users cause failures that you didn’t catch with your standard set of accelerated life tests, you can update your test plan to make it better. In certain cases, Synapse has even built specialized test equipment to mimic user failures, like our dynamic water ingress tester.
If you’ve spent months or even years working on a particular project, it can feel like you know your mission inside and out, and user testing isn’t the best use of your time. If you can humbly accept that you don’t know all the things about your user (who knew people use electric toothbrushes on their chihuahuas??) this step will feel easier, and like the wise investment that it is.
Were you as deliberate about reliability and verification testing as you were with the design?
Design engineers tend to focus on design, but testing is the step in the product development process where you really understand and improve your product. Do you have a realistic understanding of how reliable your product needs to be? Do you have a verification plan that enables you to see how well you’ve met your design goals? When you write that plan, don’t just test to your product requirements, test to failure. If a button needs to survive 10,000 presses, it’s useful to know whether it fails at 11,000 or 100,000 presses. If a sensor needs to work over an operating temperature range of 0-60C with a certain accuracy, it’s useful to know whether it goes out of specification at -5C or -30C. Design margin helps you scale products to volume production, and good verification testing tells you how much design margin you have.
Design engineers tend to focus on design, but testing is the step in the product development process where you really understand and improve your product.
Fear of uncovering a design flaw is so entrenched, even the most seasoned engineers can want to ‘ostrich’. It helps to think of each failure you see in verification testing as something you won’t have to fix once the product is shipping. Redesign is expensive, but we build it into schedules intentionally because no one designs a perfect product in the first attempt. Redesign after launch can be ruinously expensive, so use your test data to redesign at the right time.
Does your product meet regulatory requirements?
You can’t sell an electronic product without going through regulatory certification. If you’re selling a wirelessly connected product, this certification process is more complicated. Unfortunately, different countries have different regulatory certification requirements, and the process can quickly become expensive and complex. You’ll want to make sure your engineers and marketing / sales team are on the same page about launch plans. Regulatory testing also frequently gets ignored until right before launch, when the costs of redesign are high. When you get your first form factor prototypes, you should consider a regulatory pre-screen to look for issues. Regulations aren’t just for RF and EE types either. For example: if you have a project where biocompatibility is required, it is much better to consider that from the beginning, and work the testing into the schedule, rather than to have to re-run a design cycle to make parts in better or different materials.
Do you have production testing in place?
A successful pre-production build with your chosen manufacturer is a big, thrilling milestone. High fives and champagne all around! But how can you ensure that the first 10,000 units you ship will all work exactly like these prototypes? As manufacturing ramps up, it is quite common for processes to change or procedures to be streamlined. Parts get second-sourced, and operators doing the assembly change. Do you have sufficient production line tests in place to catch any issues that these changes can cause? In order to design a production test system, you’ll need to work with the manufacturer and refer to your product requirements. What are the most important things to test? Do you need to test them on every unit or are you comfortable with batch testing?
How good is good enough to ship?
By now, you might be saying “OK, OK, I get it, testing is important. But you said you’d tell me how to know when I’m ready to ship.” We’ve spent so long talking about testing because that’s how you understand where you are relative to “done”. It’s very likely that you’ll go through the testing we describe and find out your product isn’t perfect. So what do you do then?
The reality is that imperfect products ship all the time, and sometimes shipping an imperfect product sooner is the right business decision.
The reality is that imperfect products ship all the time, and sometimes shipping an imperfect product sooner is the right business decision. If you’re running a startup that needs a product on the market to reach a funding milestone, shipping a product that meets most but not all of your requirements is probably the right answer. Maybe you’re part of a larger organization that’s trying to launch a completely new type of product and the value of getting something on the market to get real consumer insights is worth the tradeoff of implementing a no-questions-asked returns policy and replacing failed units. If you’ve done good testing, you have the tools to make an informed decision when you find yourself faced with this choice. When you get a test failure, you can go back to your product requirements for context about why the requirement was included, how critical it is, and what else it impacts. Even making a strategic decision to skip certain types of testing to save time gives you more information - you know a little bit more about what you don’t know. In the end, we can’t actually tell you when you’re ready to ship, only you can decide that based on the realities of your situation.
To engineers, it can often seem as if “done” is when the design is done - cross that finish line, and relax. In reality, the design isn’t “done” until it has been designed, tested, re-designed, re-tested, certified, scaled, and re-tested. After all that, done IS beautiful.
*Martine brought this saying to Synapse, and she got it from the Bucknell University Department of Theatre and Dance. If you want to feel better about your tight timelines, see someone who works in costumes or set design.
As the only form of public transit that is readily capable of supporting social distancing, shared bikes and scooters will be an essential component of an effective urban economic recovery strategy in the coming months. Prior to COVID-19, the city of San Francisco announced that future shared scooter platforms will be required to have deterrents to riding on sidewalks. Naturally, being both SF residents who will be affected by this decision, and curious engineers with micromobility industry experience, we decided to leverage our expertise in machine learning to explore the specifics involved in implementing a system which can determine if a scooter is riding on a street or a sidewalk.
Architecture development for innovative products is exciting because options abound and we can exercise our creativity. At the same time, uncertainty lurks in the user experience, business case, and technical realms. Learn how we navigate this challenging terrain to help our clients deliver world-class products to market.
We work with our clients to conceive and develop innovative products to transform their businesses, which often results in significant engineering challenges. If we dive into the fundamental phenomena underlying those challenges and apply solutions that directly address them, the best solution is often the simplest and lowest cost. We’ve rewired our instincts to uncover simple, but hidden solutions to complex problems, as the simple solutions are often the ones that best align with our clients’ business constraints.
"How much will it cost?" and "How long will it take?" are often the first two questions prospective clients ask when they approach us to develop their hardware product concept. To better estimate, we've laid out some of the major influencing levers that can be utilized to reach your budget and schedule targets in this ebook.
Synapse is a product development firm. We work with the best companies in the world to drive innovation and introduce cutting-edge devices that positively impact our lives. Fueled by a desire to solve complex engineering challenges, we develop products that transform brands and accelerate advances in technology.