The ultimate purpose of Product Backlog refinement is to ensure an ongoing conversation that increases transparency of the Product Backlog and therefore the Product itself – to orient everyone on the team to breaking out of their waterfall silos and focus on delivering business value, period.
On mature teams, a lot of the refinement work happens as ad hoc conversations while they are sitting around and thinking together about how to build something great because they are just motivated by that and it becomes part of their mode of operation.
The objective of the refinement work of any given Sprint (that often needs to be repeated over and over like a mantra with new, immature teams) is to ensure that the items at the top of the Backlog are transparent enough that the Development Team considers them ready to pull and get “Done” in the next Sprint. This is where the concept of the Definition of “Ready” (DoR) comes from – the Scrum Team defines the DoR and spends up to 10% of its capacity refining enough items at the top of the Backlog so that it can provide estimates (if required) and have a reasonable degree of confidence that it can deliver the items in the next Sprint.
Refinement is NOT solutioning – I think this is the big trap that a lot of teams fall into because there is a false assumption that technical solutions need to be hashed out before estimates can be made (part of the carried-over lack of trust and communication between the business and IT) – I would almost rather throw out estimates in cases where this is not improving – The Planning Game exercise, when facilitated well, lends itself more to increasing transparency rather than solutioning.
The fact that teams are telling us that they need to solution before they can estimate is also an indication of weak Agile Engineering practices such as refactoring, test-driven development and continuous integration (XP). The best refinement sessions are those in which the team is able to focus on the “what” – the business benefit results that the Product Owner really wants – rather than the “how” (solution). Strong teams emerge in an environment in which they are trusted by the business and management to find the right solution as a team. They don’t need to have it all figured out before giving an estimate because they are not afraid to give a bad estimate and fail. Also, if the team is struggling to give estimates, this is often a sign that the Product Backlog Items are too big. Most likely the team also needs to expand the Definition of “Done” to include testing against acceptance criteria within the Sprint so that they can estimate based on that criteria.
The “how” (solution) should be mapped out by the Development Team at a high level in the 2nd part of Sprint Planning (partly why the time box is bigger than they often think they need) and more detailed architecture, requirements and design work as part of the Sprint Backlog
But this level of maturity is very hard to do and it will take a while to get there, perhaps even years.
It also depends on your interpretation of “detail”, the word used in the Scrum Guide to describe what the team does in Product Backlog refinement. To me, it means understanding in more detail what the Product Owner really wants and needs. What does it mean to you?