soapUI vs. JUnit

Posted on 04/28/2007 , 12 Comments ( Add )

Open source soapUI tool has a number of benefits over developing test cases in Java using JUnit framework. Some of these benefits include:

  • Use of UI for creating/configuring tests. This makes it possible for non-programmers to develop tests, which means that testing of Web services can be performed by a QA/system test group.
  • Auto-generation of test messages based on WSDL.
  • Ability to externalize test data using property files.
  • Built-in assertions, such as checking for SOAP faults and validity of the schema.
  • Built-in stress testing capabilities.
  • Management of endpoint URLs.

Overall, soapUI allows developers and testers to create tests quicker and also makes tests easier to manage and update.

However, one needs to understand shortcomings of this approach. soapUI uses its own Web services client. The way it creates test messages is different from how regular (JAX-WS-based and other) Web services clients work. soapUI completely skips the step of generating Java classes from WSDL. It does not have to deal with XML marshalling/unmarshalling from/to Java objects. As a result, you are not invoking your service the way it will be invoked by real Web service consumers.

Technically, it may not be an issue in a majority of cases. The fact that soapUI is able to parse your WSDL/schema and generate test message means that you have a valid WSDL that should also be consumable by other technologies. However, it is possible that your WSDL/ schema may lead to sub-optimal (albeit perfectly valid) generated classes. For example, the schema that defines <addresses><address>...</address>...</addresses> XML will result in JAXB compiler generating Addresses class that returns the list of type Address. This may or may not be how you want Java classes to look like for the clients. Perhaps you may want to consider to get rid of <addresses> so that no extra class is generated.

In other words, using soapUI (or similar UI-based Web services testing tools, such as SOAPScope) does not allow you to understand all the details of how your Web service will be consumed in the real world.

Another advantage of JUnit test cases is that they can be used as examples illustrating the use of your service. The test classes can be published along with other Web service documentation or handed over to consumers to help them jump-start their client implementations. Unfortunately, soapUI test cases can't be used for this purpose.

The shortcomings of visual testing tools such as soapUI can be addressed by complementing soapUI tests with JUnit tests. I usually start with soapUI since it is so easy to get up and running with it. Once soapUI test is implemented, I go back to Java, generate Web services client classes from WSDL and implement a JUnit test class. The logic that goes into a JUnit test class obviously depends upon a Web service. But at the very least, there should be one JUnit test per service that invokes the service using generated classes and libraries provided by your Web service product of choice. This way, we can see exactly how our services will be consumed by our clients.

12 Responses to “soapUI vs. JUnit”

#7879 | April 30th, 2007 | jon

Excellent points! I enjoy your blog immensely. Keep up the great work!

#7892 | May 1st, 2007 | Ole Matzura

Hi,

nice post! I would like to add some more twists:

1) Using a tool like soapUI is mainly for testing the actual server-side web service implementation, and does give some possibilities that are not available (or at least not easy) in JAX-WS;
– sending of invalid messages (both syntactically and structurally) which is usefull for testing server-side error handling,
– sending of messages using standards not supported by JAX-WS (much of the WS-XXX soup)

2) Often when developing web services, you have little control of the actual client technology used on “the other side”, thus providing examples based on JUnit/JAX-WS could be rather limited as there are so many more technologies in use for accessing web services (but of course its better than no examples at all).
-> We have actually had several soapUI users that provide comprehensive soapUI projects together with their web services, since it is a great way of giving a “platform-neutral” example of how the web service is supposed to work. Sometimes they also include a complete Mock Implementation of the web service in soapUI, which allows their client to build/test client code without having access to the actual live service

3) Anyhow, using JUnit as you describe is a great complement and as you say gives an insight in how the web service may be “perceived” by the client. Maybe we could add support for generating stubs for these kinds of JUnit testcases for JAX-WS from within soapUI?

rock on!

kind regards,

/Ole
eviware.com

#7904 | May 2nd, 2007 | Alexander Ananiev

Ole, I agree with you on #2 in that typically service consumers would be using different types of Web services platforms, not just JAX-WS (otherwise, what would be the point of using Web services in the first place). However, from my experience, in a corporate environment (Internet is a totally different matter) these technologies are limited to mostly different flavors of Java and .NET and so there are not that many permutations that would have to be tested using XUnit approach.

On #3, I actually think that generating JUnit stubs from soapUI would somewhat defeat the purpose of developing JUnit classes, which is to allow service providers to perform all the steps that service consumers would have to go through in order to consume a service. And I’m pretty sure that you already have enough on your plate with all the other work on soapUI :)

Thanks for the great tool,
Alexander

#26119 | April 17th, 2008 | sure

Hi,

I am new to this tool, Can any one help me how to test webservice using this tool,

#34975 | June 4th, 2009 | July

Hello,
I need your help…………. I am trying to connect a web service, but it doesn’t work using JAX-WS API, but when I test it with the soapUI tool it works, so I want to know how soapUI works……………it is very important, please is someone can help me?

Thanks in advance,
July

#35024 | August 4th, 2009 | Krishna

I am new to SOAPUI and want to know how to invoke API from SOAPUI
Can you please let me know, what steps need to be followed to invoke the API?

#35109 | November 30th, 2009 | Anand

Good Post..I got exactly what I want. This saved at least an hour for looking around.
I follow all your posting in SOA section as well..
Many Thanks

#35221 | March 2nd, 2010 | Sourabha

I have done schema validation for synchronous services using SOAP UI. i tried doing validation of asynchronous services but i could not figure it out properly. i would also like to know how we do load and stress testing using this tool.

#35364 | October 14th, 2010 | bart

To use SoapUI for testing is not the best idea.
It has support for test cases and test suites, but the setup is extremely time-consuming.
Also, when compared to a JUnit test, it takes about twice as long to setup a test in SoapUI as it does to write the same JUnit test, and the JUnit test will be testing the ws client code.
SoapUI has some serious usability issues. Try to create a test that uses 3 services, where the first service is a login that returns a session id to be used for the other 2 services. It takes about 2-3 hours to create that in SoapUI and about 1 hour in JUnit, including the time to generate the client code from the WSDL.

#35388 | June 6th, 2011 | JUnit vs. soapUI – Websphere Tutorial

[…] good post: JUnit vs. soapUI: Open source soapUI tool has a number of benefits over developing test cases in Java using JUnit […]

#35522 | November 20th, 2011 | small business website promotion

small business website promotion…

[…]soapUI vs. JUnit[…]…

#37136 | October 28th, 2013 | Aditya

could you please provide an example to write junit test case at the endpoints for java stubs generated from wsdl?