Lettuce and Cucumber
For those that don’t know, Lettuce and Cucumber are different frameworks that utilize BDD (behavior driven development.) BDD is a form of greater specification, which works well for test engineers and QA Automation specialists.
Tests are written in a human readable format, such as:
Given a visitor to our websites homepage
When they register as a new user
Then their registration is saved to the database
And the user is redirected to their new dashboard
This is easy to read, understandable and concise. When a test fails, you get a nice clear failure message and a count of how many tests have passed/failed and what step was the failure.
For many automation frameworks, I’m a big fan of BDD. It’s not necessary for every framework or tool, but for things like web automation (where you have many moving parts that need lots of tests covering edge cases and dozens/hundreds of scenarios) it’s quite useful to make sure you have enough coverage. Anyone in the business unit can easily read the tests and tell you if you are missing something or not.
So why Lettuce? I’ve recently been learning Python and thought it would be nice to check into their own BDD frameworks. Cucumber doesn’t port to Python yet, so I looked into the alternatives. Lettuce looks (on the surface) just like Cucumber.
However, once I started peeling back Lettuce, it started looking pretty bad. I had a lot of the same frustrations I’ve had with other BDD frameworks (i.e. GEB.) Which make Lettuce feel unfinished and unpolished.
If you have a system where you’re using multiple versions of Python, Lettuce becomes a pain. I installed it on both my Python 2.7* as well as my Python 3 versions. Ok. So now when I want to run Lettuce, how can I do it? With Cucumber you can run it right in your IDE (i.e. in RubyMine for example.) You can right click any Cucumber scenario in the IDE and just run that scenario. Nice and easy.
Not so with Lettuce. JetBrains’ Python IDE (PyCharm) doesn’t have the ability to run a specific scenario. I couldn’t find an IDE that would. That means I have to run Lettuce from the command line…. So if my IDE is using Python3, and my system defaults to Python 2.7.*, then I have to go and change my system’s python to run this. Something I don’t have to do in Cucumber.
Cucumber uses the concept of steps (where you put the code to drive the Feature’s Given/When/Then.) This keeps the Feature file from getting cluttered with code. It always remains human readable.
The steps are mapped with Regex. So if you have a Feature scenario like:
Given a user goes to “www.google.com”
Then you would have a step like this in Cucumber:
Given(/^a user goes to “([^”]*)}$/) do |arg|
some code goes here
In Lettuce it would look like:
@step(‘a user goes to (.*)’)
def user_google(step, url):
world.string = url
My problem with the lettuce steps is that it’s not clear what’s going on. What’s this world thing? Like GEB, it’s an attempt to make a better Cucumber, but I don’t get it’s reason for existing. Cucumber passes the variable just like a Ruby method. If you know Ruby, you get Cucumber. If you know Python, would you get t his world concept?
Why is there an annotation for the step (with the regex) and then a def under it defining what the step itself? I find it redundant. In Cucumber the object being passed into the step is already typed via regex for what it is. For example, if I had a step:
@step(‘our number is (\d+)’)
I am setting the type coming in as a int. Why do I need to write:
def some_value(step, number):
under the step annotation? It just seems redundant.
Step File Locations
In Cucumber the step files all go into a sub folder of the Feature directory called “steps.” This leaves just the features in the Features directory. It’s clean.
In Lettuce, you have to put all your steps as .py files in your Features folder. So your feature files and steps are intermingled. I feel it looks messy. I don’t like mixing the steps and features together, especially if non tech people might be reviewing the BDD’s… I don’t want them clicking into a step file by accident and getting confused.
Lack of IDE support
Similar to not being able to run Lettuce scenarios and features from the IDE, you also can’t generate steps from the IDE’s.
With a Ruby IDE like RubyMine, I can right click a Feature step (like Given “a user goes to google.com”) and choose a menu item to create this step for me.
It will auto create the step file, and put the regex needed to link the Feature step to the step definition! How cool is that? Sometimes the regex is complex, so I don’t want to write it out by hand each and every time.
In PyCharm (Python IDE) I can’t do that. If I right+click a Feature step, the only options I have are to Edit or Suppress it from being called! So you can’t generate the step itself. That auto generation saves a lot of time in Cucumber.
All this and then Lettuce just didn’t behave for me. I ran it and got some weird errors about exception handling. It could be that I had to run from a command line and my default python is 2.7* and the Lettuce project uses python3… I’ll have to dig deeper.
My theory on this stuff is simple: If it doesn’t work easily, I don’t use it. I’m a consumer of libraries. I don’t want to sit and “make” this work for me. Cucumber Works. I don’t have to sit there and go “oh I’m using Ruby 1.9.3 vs 2.1 in my project.” I can still run Cucumber in the IDE. Or, I can easily switch Ruby with RVM… I have tried using virtualenv within Python to set up environments to work with lettuce… but it’s a PITA.
I’ve met so many people who run from Cucumber seeking an alternate… and I just don’t get it, as their alternates just don’t work! GEB is a fine example of something attempting to replace Cucumber, but a poor imitation of it. I totally understand if your stack is in a language and you want to match that language… i.e. you have a web app written in Django and you want to test it in Python. But if you don’t need to be language compatible, why go down the road of these beta versioned BDD frameworks that are really a challenge to get working?
At any rate, I love Python, but this Lettuce leaves a lot to be desired.