Implementing UI Test Automation with SpecFlow and Gherkin for Effective Software Testing

UI Test Automation with SpecFlow Presented by Eugeny Govako For internal use only

Overview Gherkin and test automation NUnit based automation approach SpecFlow automation approach SpecFlow automation evolution Q&A For internal use only

Gherkin and test automation Test Automation is implementing specifications step by step using some programming language Gherkin is the language for executable specifications Gherkin itself is a kind of a programming language with well defined syntax Originally invented to use with Cucumber framework For internal use only

Prerequisites Specifications (written in Gherkin preferably) Test API Driver providing access to Application with ability to find required control Wrappers for controls Application under test Gherkin Test TestAPI Application For internal use only

Non-SpecFlow automation NUnit tests implementation based on Gherkin lines Scenario: 8741_01 Red button should turn on green light Given red button is visible When user presses red button Then Green light should be shown [Test] public void RedButtonShouldTurnOnGreenLight() { // Given red button is visible MyScreen.RedButton.IsVisible.ShouldBe(true); // When user presses red button MyScreen.RedButton.Click(); // Then Green light should be shown MyScreen.GreenLight.IsShown.ShouldBecome(true); } For internal use only

Non-SpecFlow automation summary No direct link between specification (Gherkin) and test code Test code uses Test API layer that provides access to controls, allowing application interaction Gherkin Test TestAPI Application For internal use only

SpecFlow automation Specification linked to test Test development is very fast Scenario: 8741_01 Red button functionality Given red button is visible When user presses red button Then Green light should be shown Automation code base grows even more fast Test Support trouble [Given(@"Red button is visible")] public void RedButtonShouldBeVisible() { MyScreen.RedButton.IsVisible.ShouldBe(true); } [When(@"user presses red button")] public void ClickRedButton() { MyScreen.RedButton.Click(); } [Then(@"Green light should be shown")] public void RedButtonShouldWork() { MyScreen.GreenLight.IsShown.ShouldBecome(true); } For internal use only

SpecFlow automation: basics Using parameters to reduce code amount Scenario: 8741_01 Red button functionality Given red button is visible When user presses red button Then Green light should be shown [Given(@"(.*) button is (visible|invisible)")] public void CheckButtonIsVisible(ButtonName button, ElementState state) { GetButton(button).IsVisible.ShouldBe(state.ToBool()); } [When(@"user presses (.*) button")] public void PressButton(ButtonName button) { GetButton(button).Click(); } [Then(@"(.*) light should be shown")] public void CheckLightVisibility(ControlName light) { GetControl(light).IsShown.ShouldBecome(true); } For internal use only

SpecFlow automation: basics Tables can be used row-by-row Then Client combobox should contain following list of values: | Client | | Rich client | | Common client | [Then(@"(.*) combobox should contain following list of values:")] public void CheckComboboxValues(FieldName field, Table fieldNames) { var expectedItemsList = fieldNames.ValuesToList(); var combobox = GetComboBox(field); var actualItemsList = combobox.GetDropdownItems(); actualItemsList.ShouldContainAll(expectedItemsList); } For internal use only

SpecFlow automation summary Tests code is directly connected to Gherkin via SpecFlow runner Test API layer still used for providing controls Binding method arguments reduce code amount SpecFlow engine Gherkin Test API Application Bindings Parameters For internal use only

Gherkin requirements for automation Reuse steps Use parameterization where possible Any Gherkin line can be automated. However, there s always tradeoff between effort needed to create scenario and effort needed to automate. For internal use only

SpecFlow automation: advanced features Using step argument transformations Scenario: 8741_01 Red button functionality Given red button is visible When User presses red button Then Green light is shown [Given(@"(.*) button is (visible|invisible|not visible)")] public void RedButtonShouldWork(ButtonName button, bool state) { GetPortletButton(button).IsVisible.ShouldBe(state); } [StepArgumentTransformation(@"(visible|invisible|not visible)")] public bool IsVisibleConverter(string value) { return state == "visible"; } For internal use only

SpecFlow automation: advanced features Tables can be used in transformations Tables can be converted to parameter maps Given MyScreen is opened And user fills fields with following values: | Client | Product | Quantity | | Demo Client | Demo Product | 5 | [StepDefinition(@"user fills fields with following values:")] public void FillFieldsOnScreen(Table table) { var params = table.CreateInstance<ScreenParameters>(); FillField(FieldName.TradeFor, params.TradeFor); FillField(FieldName.Strategy, params.Strategy); FillField(FieldName.CcyPair, params.CcyPair); } public class ScreenParameters { public string Client { get; set; } public string Product { get; set; } public int Quantity { get; set; } } For internal use only

SpecFlow automation: generic approach Trouble? Parameterization leads to huge maps of controls Conflicting Regex for same actions in different contexts Code base still grows fast How? Execution Context specify where to look for control Use direct transformations to automation wrappers Register controls with friendly names in a context-specific resolver Silver Bullet? Makes it possible to have bindings provided from the box, along with control wrappers For internal use only

SpecFlow automation: control-based Example: working with buttons IButton wrapper Provides access to Visibility, enabled state, text, click functionality Implementation comes from automation framework Visual Resolver: Allows registering button by friendly name Allows resolving button in an active context Automatically transforms control requested by friendly to DSL interface used inside binding method StepArgumentTransformation returning IButton Using Visual Resolver, convert friendly name (string) into IButton wrapper instance Button Bindings extending ControlBindingBase<IButton> Contain bindings that use string -> IButton conversion For internal use only

SpecFlow automation: control-based Visual Resolver example internal sealed class QuickOrderEntryPopupVisualResolver : VisualResolver <IQuickOrderEntryPopup> { public QuickOrderEntryPopupVisualResolver() { RegisterChild<IButton>("Cancel Order", AutomationIds.CancelOrderButton); RegisterChild<IButton>("Bid and Offer", AutomationIds.BidAndOfferButton); RegisterChild<IButton>("Bid", AutomationIds.BidAndOfferButton); RegisterChild<IButton>("Offer", AutomationIds.BidAndOfferButton); RegisterChild("Rate", dsl => dsl.RateField); RegisterChild<ITextBox>("Amount", AutomationIds.AmountInputField); } } For internal use only

SpecFlow automation: control-based Control binding class example [Binding] internal sealed class LabelBindings : ControlBindingBase<ILabel> { public LabelBindings([NotNull] IVisualResolverManager resolver, [NotNull] IContextStack contextStack) : base(resolver, contextStack) { } [Given(@"(.* label) is (shown|not shown)")] [Then(@"(.* label) (should|should not) be shown")] public void LabelIsVisible(ILabel label, bool visibility) { Guard.ArgumentIsNotNull(label, () => () => label); label.IsVisible.ShouldBecome(visibility, Timeouts.Medium); } } } For internal use only

Control-based approach: summary Uses SpecFlow native step interpreter Uses Regexp for steps and parameters mapping Uses Visual Resolver to convert names to DSL Reduced number of steps Development costs are low Diagnostics is not good Similar steps still exist for different control types For internal use only

SpecFlow automation: behavior driven Control based example: Given My Favourite label is shown Given My Favourite label is shown Given My Favourite button is shown Given My Favourite combobox is shown Given My Favourite window is shown Scenario Block 4 binding methods Control type Control friendly name What to check Behavior driven example HOW Given My Favourite label is shown WHAT Control wrapper from Visual Resolver Behavior control should implement For internal use only

Behavior-driven approach Behavior shown / not shown enabled / disabled has some text Interface ICanBeVisible ICanBeEnabled IHasVisibleText Property IsVisible IsEnabled Text Works with all controls supporting behaviors designated by atomic interfaces New behaviors can be introduced easily public interface IMyControl : ICanBeVisible, ICanBeEnabled { ... } internal sealed class MyControlDsl : IMyControl { ... } For internal use only

Behavior driven approach: conclusion Automation bindings number significantly reduced Focus on behavior, not control (closer to business) Still risks of conflicts hard to resolve via regexp For internal use only

Romashka Driver Give up Regexp! Use smart step parser and grammar rules Find controls and analyze/apply behaviors No Regexp conflicts New behaviors added easily Support new controls by applying behavior interfaces to DSL Diagnostics is even worse Hard to understand and contribute Crush a fly upon a wheel feeling For internal use only

Specflow automation in Autobahn FX UI 100% P1 / 80% all automation coverage (of possible to automate) in 5 projects using SpecFlow More than 23000 test cases automated Reasonable test support costs For internal use only

Q&A For internal use only