Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

Behavior-Driven Development with its emblematic “Given-When-Then” scenarios syntax has lately been quite hyped for its possible uses as a boundary object for software functionality assessment.

I definitely agree that Gherkin, or whichever feature definition script you prefer, is a business-readable DSL, and already provides value as such.

However, I disagree that it is writable by non-programmers (as does Martin Fowler).

Does anyone have accounts of scenarios being written by non-programmers, then instrumented by developers?

If there is indeed a consensus on the lack of writability, then would you see a problem with a tool that, instead of starting with the scenarios and instrumenting them, would generate business-readable scenarios from the actual tests?

Update: regarding the “scenario-generator” tool, it would of course not guess business language magically  ;)  But, just like we currently use regexp matchers to create tests in a top-down approach (on the abstraction dimension), we could use string builders to create scenarios in a bottom-up approach.

A “to give an idea only” example:

Given I am on page ${test.currentPage.name}
And I click on element ${test.currentAction.element}
…
share|improve this question
    
It will be long time before humans will be able to come with common language readable by other humans in exact way, even after computers will already be able to write code for computers. –  user7071 Jun 5 '12 at 22:34
    
Ironically, JBehave 1 (the first BDD tool) started by generating business-readable scenarios. We didn't parse English until Cucumber. I think JBehave 1 was useful for actually reminding people that they had to talk about them first... –  Lunivore Jun 23 '13 at 17:29

4 Answers 4

up vote 18 down vote accepted

I have seen it. Didn't end up well.

I think that cucumber is cumbersome (<--lol :D) abstraction for this exact reason. Too hard for non-technical people to write by themselves; too verbose for technical people.

Non technical people just haven't learned to think like programmers. It is our privilege to understand abstract knowledge, break it down, build up anew and still be able to run away from ambiguousness successfully. That's what we are paid for.

If there is indeed a consensus on the lack of writability, then would you see a problem with a tool that, instead of starting with the scenarios and instrumenting them, would generate business-readable scenarios from the actual tests?

Tool itself won't be able to generate them. Computer knows nothing about business domain. In the end - programmer would be responsible for drawing what needs to be generated anyway and even then - likely those scenarios would be too verbose/cryptic for their end-users.

share|improve this answer
15  
Non technical people just haven't learned to think like programmers. The truth. This same concept has been hyped and reinvented a number of times in the past 20 years and it almost always ends up with poor results. Businesses like the concept of getting software without having to pay those greedy blood sucking leech software developers but they forget that the hardest part of software development is most of the time understanding the business rules deeper and more intricately than the business guys themselves. –  maple_shaft Jun 5 '12 at 15:14
2  
“Non technical people just haven't learned to think like programmers.” Yep, the ability to decompose problems and express them within specified atomic logical terms is probably what makes one a programmer / analyst. The idea that looking like English makes Gherkin usable by anyone seems incredibly naive to me. Thanks for confirming it :) Computer knows nothing about business domain. Of course. I did not make my idea very clear, sorry about that. I will add info to the question. –  MattiSG Jun 5 '12 at 17:02
6  
@maple_shaft: The past 20 years? Try the past 60 years. Some of the early hype around COBOL was that business people could write it, eliminating the need for programmers. When that failed to happen, people came up with a bunch of what they called fourth-generation languages that were supposed to do the same thing. –  David Thornley Jun 5 '12 at 21:19

Part of the difficulty in terms of the customer writing a specifications document is that the customer often doesn't know how to translate the things the customer wants into a language which actually describes what the customer needs. While the customer may say that they want a certain behaviour to exist in a system, they are generally not so concerned with the minutiae until they have seen and used and experienced the software working in a manner which the customer feels doesn't quite match their needs.

When customers describe a business process, they often leave a lot of relevant information out. Often this information relates to things about a process that are commonly understood within the customer's particular domain and which is therefore taken for granted and often not relayed to the programmer. At other times, the customer doesn't actually know how to deal with all of the boundary conditions within a system, and is looking to the programmer for guidance. Sometimes it's all a simple case of usability, with the customer thinking they want something to work in one way, but later changing mind when it becomes clearer that things should work differently.

Ok, so enough of "customer-relations 101 for programmers". The question is whether there is still value in having the customer use a business readable DSL to define how to define a specification. I believe that with guidance, the answer is a tentative 'yes', and I say tentative because the next question that comes to mind is, why would you have a customer craft a DSL when you can have a programmer more easily define one that will provide a customer with a simple yet rich language to define how a system needs to work?

When you have provided a customer with a language for describing how they would like a system to work, you are going to end up with statements that say something along the lines of:

"for a given 'subsystem', as a 'business entity' I want 'some feature' so that I might achieve 'some result'".

This type of statement ends up describing a requirement in a very clear way, providing the the overall shape that the customer basically wishes the system to assume, or another way of looking at it is that the customer is describing what the system is. If you wish to have your customer think about things a little further, you can then ask them to describe the rules which the feature must obey using a number of statement similar perhaps to:

"Given 'some system state', When 'some action occurs', Then expect 'some result'

Again very clear descriptions, this time about how the system should behave. The thing is, this will not replace the need for a software developer to fill in all of the blanks, and to tease out further details that the customer may be only peripherally aware of. While the customer may be able to be 'trained' by the programmer to describe features and behaviours in a nice programmer-friendly format, the customer won't really have the skills or the knowledge to generate meaningful test cases, nor to provide the implementation code. This was I think the point of the Martin Fowler article the OP has referred to. So yes, the software itself isn't writable by the customer, but the description of the software most certainly can - and IMHO should - be written by the customer. For what it's worth, I didn't read Fowler's article as saying that the customer shouldn't or can't write the DSL, but rather that the cost in creating a tool that would ensure the customer can understand how to successfully implement a DSL is effectively prohibitive, which is why I feel it's easier and perhaps more cost effective to show the customer a DSL, and help the customer tune it if needed in order to provide a common language which both business people and programmers can use to collectively stay on the same page.

I feel that we programmers can sometimes forget that our customers are generally very smart in terms of their understanding of their businesses and business processes, certainly much better than we do. When they don't have a programmer to tell them how to build a software system, the customers generally resort to other - perhaps less efficient - means to solve their particular business management problems. By this I mean simple databases (think Access) or spreadsheets, or even in hand written ledgers, and with well defined rules and procedures to manage those processes. What many customers lack isn't a means to determine how a system needs to work, but rather how it should be built, and more importantly how efficiently describe the behavioural rules of a system to the people who do have the skills to actually build the system.

If there is indeed a consensus on the lack of writability, then would you see a problem with a tool that, instead of starting with the scenarios and instrumenting them, would generate business-readable scenarios from the actual tests?

I think that this is looking at the the issue the wrong way around. I would see a large problem with a tool that generates documentation from tests if that documentation was intended to represent a specification in any way. In order to test a scenario, you need to understand it, therefore the scenario needs to already exist for you to both define a test for it. If you describe the scenario in a BDD-syntax, then you have already specified it, and thus you can only instrument the scenarios after the fact. If on the other hand you had a tool that would allow the customer to describe a system in a nice programming-friendly DSL, and if that tool could be used to generate the code templates that would be used as a test suite, then I'd say there would be great value in such a tool. It would not see the customer taking programmers out of the equation, and it would help reduce the effort required to take the customer's wishes and generate test-encoded requirements in a BDD fashion, and would perhaps make the customer's wishes more easily understood. It would not however be a substitute for having an experienced software developer on hand to help the customer to separate the customer's needs from the customer's wants.

share|improve this answer
    
“In order to test a scenario, you need to understand it, therefore the scenario needs to already exist for you to both define a test for it.” I agree. What I am questioning is whether enforcing language constraints is worth anything. I don't claim we should write only tests; but I wonder whether we shouldn't accept the fact that business description will (and probably should) always be plain, free form descriptions. Hence, we'd have pure business descs, and generated readable scenarios, letting humans to decide whether they match; rather than pretending we use actual descs to test. –  MattiSG Jun 6 '12 at 7:56
    
@MattiSG ...whether enforcing language constraints is worth anything. This is a good question. Free-form descriptions are more expressive and natural to the writer, but result in rambling commentary that requires dissection in order to derive a useful spec. By defining formal 'rules' (aka a DSL) for writing requirements and specs, you have a common language that both customer and programmer can understand, limiting misunderstanding. If you get the descriptions right, they can be used verbatim as a template for your tests. Thus no need for complex tools to "generate" anything. –  S.Robins Jun 6 '12 at 10:36
    
@MattiSG FWIW, using a DSL and BDD is the system I myself use. Requirements are defined as Entity-Feature-Benefit statements, and followed up with a spec which extend initial requirements statements using AAA (I.e: Given-When-Then) statements... essentially scenario statements. The difficulty when attempting to decode free text is that without a DSL, you don't have an easy means to define an algorithm that can generate meaningful collection scenarios. My point was that using the tests as a starting point to generate scenarios is kind of backwards. –  S.Robins Jun 6 '12 at 11:56

I've seen developers write scenarios; testers write scenarios and even a product owner write scenarios. I've also had conversations explicitly designed to bring out scenarios - "Given <this other context>, when what should happen then?" - and written down the words the business use.

The best results I've had were from having a conversation with the product owner while he was in the office, capturing them on a wiki then sending them to him so he could correct and add more. He found a couple we'd missed in our conversations. That rocked.

The worst scenarios I've seen are the ones developers wrote themselves without any conversation with the business. I can tell because they use terms like "When I perform a search", or "When I submit the form with a date of today + 3". They're usually not very interesting scenarios, far too many of them, too low a level of detail and therefore completely unmaintainable. The business also don't read these.

Much better to focus on the conversations IMO. I've seen a few teams do this now for a couple of months with huge improvements to quality, irrespective of the automation. One team managed to get automation working in a very difficult environment some weeks later, much to the joy of the tester! But really, as long as the team are having the conversations and using scenarios to draw out other scenarios, I don't think it matters who writes them down.

share|improve this answer
    
+1 Communication really is the key, and the scenarios really do need to be in the terms that the business people us, so in keeping with the OPs question, if we create a DSL, this really needs to be able to be a closer match to what the customer is going to say, and not what the programmers think the customer should be saying. –  S.Robins Jun 14 '12 at 23:47

My experience is that it is best left to the BA to write the GWT(Given-When-Then)s(experience using SpecFlow). The BA can translate the Customer requirements into the GWT and the business can read it. Customers understand the systems but they do not have the tech think to write the requirements in a way that we can use.

Ideally the BA would write some GWT, I would make some revisions, the BA would review/revise repeat until the BA and business was confident in the coverage. Practically the BA would give me a rough draft that I would clean up and make work. The Business shrugs it off saying sure then wonders why it didn't cover some areas no one thought of.

share|improve this answer
    
Could you please explicit what GWT means to you? :) –  MattiSG Jun 6 '12 at 7:19
    
@MattiSG: guess it's for Given-When-Then (see OP). –  sleske Jun 6 '12 at 12:41
    
@MattiSG - Updated the post good catch. –  Chad Jun 6 '12 at 13:00

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.