Verification

General

The verification in Atata consists of assertion, expectation and waiting. In order to verify a component or a value property, use the following properties:

  • .Should.* - an assertion. For example: UserName.Should.Be("John"). If an assertion fails, throws an assertion exception.
  • .ExpectTo.* - an expectation, which produces a warning in the end of a test. Many warnings can be collected together.
  • .WaitTo.* - a waiting for a certain condition. For example: Header.WaitTo.BeVisible(). If a waiting fails, throws a timeout exception.

Retries

A verification works with retries until a condition is met or time is out. AtataContext.VerificationTimeout property is used with 5 seconds as the default value. AtataContext.VerificationRetryInterval property is used with 500 milliseconds as the default value.

Without Retries

To execute an assertion without retries, use AtOnce property:

page.SomeInput.Should.AtOnce.BeVisible();

With Specific Retry Timeout

To execute an assertion with timeout (and/or retry interval) different from default one, use of one the following methods:

public TVerificationProvider Within(TimeSpan timeout, TimeSpan? retryInterval = null);

public TVerificationProvider WithinSeconds(double timeoutSeconds, double? retryIntervalSeconds = null);

public TVerificationProvider WithRetryInterval(TimeSpan retryInterval);

public TVerificationProvider WithRetryIntervalSeconds(double retryIntervalSeconds);

For example:

page.Component.Should.WithinSeconds(30).BeVisible();

Negative Verification

In order to execute negative verification, use Not:

page.TextInput.Should.Not.BeEmpty();
page.SomeLost.Should.Not.Contain("some item");

Ignore String Case

In order to ignore string case during verification, use IgnoringCase property:

page.SomeInput.Should.IgnoringCase.Equal("some value");
page.SomeStringList.Should.IgnoringCase.EqualSequence("a", "b", "c");

Equality Comparison

It is possible to customize the comparison of objects, mostly complex objects. Also this can be used for approximate comparison of values.

The base VerificationProvider<TVerificationProvider, TOwner> provides the following method:

public TVerificationProvider Using<T>(IEqualityComparer<T> equalityComparer);

Usage

page.SomeNumberInput.Should.Using(new CustomNumberEqualityComparer()).Equal(10.5);

Assertion

An assertion in Atata is done using Should property of UI component or value provider property.

Assertion works with retries until a condition is met or time is out. AtataContext.VerificationTimeout property is used with 5 seconds as the default value. AtataContext.VerificationRetryInterval property is used with 500 milliseconds as the default value.

Usage

Go.To<SomePage>()
    .SomeHeader.Should.BeVisible()
    .SomeText.Should.Be("Hello")
    .SomeParagraph.ComponentSize.Height.Should.BeGreater(50);

Configuration

The following AtataContextBuilder methods are for configuring an assertion functionality:

public AtataContextBuilder UseVerificationTimeout(TimeSpan timeout);

public AtataContextBuilder UseVerificationRetryInterval(TimeSpan interval);

public AtataContextBuilder UseAssertionExceptionType<TException>();

public AtataContextBuilder UseAssertionExceptionType(Type exceptionType);

public AtataContextBuilder UseNUnitAssertionExceptionType();

Expectation

This functionality is similar to the meaning of “soft assertions” term and provides warnings.

If the test you are writing is quite complex and you need to do several non-critical assertions during the test flow without breaking the test with an assertion exception when the first assertion fails, the expectation functionality is for the help.

An expectation doesn’t throw an exception on failure, but it writes a warning assertion result to the log and adds it to AtataContext.Current.PendingFailureAssertionResults collection. When on TearDown AtataContext.Current?.Dispose() is invoked, after the actual cleaning it will throw aggregate assertion exception with all found expectation failures. When using NUnit (AtataContextBuilder.UseNUnitWarningReportStrategy()), it will record every expectation failure as NUnit warning, which is similar. It is also absolutely valid to use expectations inside AggregateAssert sections.

Usage

The usage is similar to assertions. Just use ExpectTo instead of Should:

Component.ExpectTo.BeVisible()

Configuration

The following AtataContextBuilder methods are for configuring an expectation functionality:

public AtataContextBuilder UseVerificationTimeout(TimeSpan timeout);

public AtataContextBuilder UseVerificationRetryInterval(TimeSpan interval);

public AtataContextBuilder UseWarningReportStrategy(IWarningReportStrategy strategy);

public AtataContextBuilder UseNUnitWarningReportStrategy();

When using NUnit, it is available to invoke UseNUnitWarningReportStrategy() during AtataContext configuration to record warnings as NUnit’s built-in warnings. For other testing frameworks (xUnit, MSTest) the native Atata expectation functionality will work well by default.

Waiting

The waiting functionality basically does the same as assertion does, but it uses WaitingTimeout and WaitingRetryInterval property values of AtataContext, and when a waiting fails, it throws a TimeoutException.

Usage

The usage is similar to assertions. Just use WaitTo instead of Should:

Component.WaitTo.BeEnabled()

Component.Content.WaitTo.Be("...")

If a condition is not met during the waiting time, which is taken from AtataContext by default (AtataContext.Current.WaitingTimeout and AtataContext.Current.WaitingRetryInterval), then TimeoutException is thrown.

Configuration

The following AtataContextBuilder methods are for configuring a waiting functionality:

public AtataContextBuilder UseWaitingTimeout(TimeSpan timeout);

public AtataContextBuilder UseWaitingRetryInterval(TimeSpan interval);

Aggregate Assertion

Aggregate (multiple) assertion functionality allows execution of several Should.* assertions, collect the failed results and produce a single exception containing a set of failed assertions. Each failed assertion in scope of aggregate assert is also written to log.

There are 2 implemented strategies/approaches of aggregate assertion:

  • Native - default Atata approach. Should be used when you don’t use NUnit with Atata.
  • NUnit - uses Assert.Multiple and other related methods of NUnit to provide a NUnit’s assertion handling.

Custom approach can also be implemented by implementing IAggregateAssertionStrategy.

Usage

AggregateAssert methods are present in PageObject<TOwner> and AtataContext.

Aggregate Assert Page Object

Go.To<SomeUserPage>()
    .AggregateAssert(x => x
        .ContactDetails.FirstName.Should.Equal("John")
        .ContactDetails.LastName.Should.Equal("Smith"));

Or:

Go.To<SomeUserPage>()
    .AggregateAssert(x =>
    {
        x.ContactDetails.FirstName.Should.Equal("John");
        x.ContactDetails.LastName.Should.Equal("Smith");
    });

Aggregate Assert Component

Go.To<SomeUserPage>()
    .AggregateAssert(x => x.ContactDetails, x =>
    {
        x.FirstName.Should.Equal("John");
        x.LastName.Should.Equal("Smith");
    });

Aggregate Assert Block

AtataContext.Current.AggregateAssert(() =>
{
    Go.To<SomeUserPage>()
        .ContactDetails.FirstName.Should.Equal("John")
        .ContactDetails.LastName.Should.Equal("Smith");

    // Do other assertions...
});

Configuration

The following AtataContextBuilder methods are for configuring an aggregate assertion functionality:

public AtataContextBuilder UseAggregateAssertionExceptionType(Type exceptionType);

public AtataContextBuilder UseAggregateAssertionExceptionType<TException>()
    where TException : Exception;

public AtataContextBuilder UseAggregateAssertionStrategy(IAggregateAssertionStrategy strategy);

public AtataContextBuilder UseAggregateAssertionStrategy<TAggregateAssertionStrategy>()
    where TAggregateAssertionStrategy : IAggregateAssertionStrategy, new();

public AtataContextBuilder UseNUnitAggregateAssertionStrategy();

Apply NUnit Aggregate Assertion Strategy

When using NUnit, it is recommended to invoke UseNUnitAggregateAssertionStrategy() or UseAllNUnitFeatures() during AtataContext configuration to enable NUnit’s built-in multiple assert functionality. For other testing frameworks (xUnit, MSTest) the native Atata assertion functionality will work well by default.

AtataContext.GlobalConfiguration
    .UseNUnitAggregateAssertionStrategy();

Or better:

AtataContext.GlobalConfiguration
    .UseAllNUnitFeatures();