Hold your horses.
I said, don't worry. Really, there is nothing to worry about. Why? Because you took your time to refine well. You know what to build. You spoke about it and asked questions about it. You even gave a suggestion to do it a little bit different and while doing so, the customer would be better served. You developers have investigated the code by getting a resource. Someone who knows how the current situation is. He gave you anchors where to begin and how the architecture was build up and why decisions where made.The product owner has explained what the customer wants and why he or she wants it. You know the benefits of the solution. You all together created some scenarios, just to get a good feeling about it. And those scenarios helped you to find a direction to go to and which way you should not. You triggered someone to say something no one had thought about. You eliminated the questions and a whole lot of uncertainties.