Skip to main content

Ordering tests

4 replies [Last post]
palineo
Offline
Joined: 2009-02-26

Hello,

I am wondering if is there is a simple way to order test while launching a test suite ?

(for example when I launch a bundle with 10 tests, they are all done without any order).

Thank you,
Brice.

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Vladimir Sizikov

Hi Bruce,

Thanks for your interest!

Let me clarify the situation first. Why would you want to sort the test
execution in the first place? Ideally, and especially for TCK-like test
suites, all the tests should be independent entities, and should be able
to run in any order...

The JTharness typically dispatches tests for execution from the top of
the test tree moving down. So, that's what typically users expect.

If you have such tests that depend on each other, this is *very* tricky
and error prone and my suggestion would be to eliminate such unhealthy
:) dependency.

Thanks,
--Vladimir

On 4/2/2009 8:54 AM, meframework@mobileandembedded.org wrote:
> Hello,
>
> I am wondering if is there is a simple way to order test while launching a test suite ?
>
> (for example when I launch a bundle with 10 tests, they are all done without any order).
>
> Thank you,
> Brice.
[vladimir_sizikov.vcf]
---------------------------------------------------------------------
To unsubscribe, e-mail: meframework-unsubscribe@cqme.dev.java.net
For additional commands, e-mail: meframework-help@cqme.dev.java.net

palineo
Offline
Joined: 2009-02-26

Hi Vladimir,

Thank you for your quick response.

In fact I do not have any depends between test (ouff...) but my client tell me 'hummm it should be a good thing if you name tests with a prefix according to test order at runtime' (something like T01_JSR....).

I did it, but in fact each run as its own order, I do not retrieve this
=> 'The JTharness typically dispatches tests for execution from the top of
the test tree moving down.'

Simplest solution is to suppress this prefix and explain why.

Thanks,
Brice.

Vladimir Sizikov

Hi Bruce,

> In fact I do not have any depends between test (ouff...) but my client
> tell me 'hummm it should be a good thing if you name tests with a prefix
> according to test order at runtime' (something like T01_JSR....).

Hmmm, in many, many TCKs we've done in the past, we have never seen any
requests from users to have specific order of test execution.

But you could organize your tests in such a way that users of your test
suite would feel comportable.

See below.

> I did it, but in fact each run as its own order, I do not retrieve this
> => 'The JTharness typically dispatches tests for execution from the top of
> the test tree moving down.'

We typically organize tests in a test tree. The root of the test tree is
the name of the test suite, below that we have "sub-folders", something
like this:

JSR_01
- api
- interactive
- ota

In api section, we organize tests by packages, so that users might want
to execute only the tests for specific package.

If I understand your situation right, you have more than one JSR, or at
least have multiple sections of different tests?

You could put different test types into separate folders, or you could
put test related to different JSRs to different folders.

The test harness that, when you press Start button, will start from the
root of the test tree, and will execute tests from top to bottom of your
tree.

Let me know if I totally misunderstood your question... :)

Thanks,
--Vladimir
[vladimir_sizikov.vcf]
---------------------------------------------------------------------
To unsubscribe, e-mail: meframework-unsubscribe@cqme.dev.java.net
For additional commands, e-mail: meframework-help@cqme.dev.java.net

palineo
Offline
Joined: 2009-02-26

Hi Vladimir,

You understood my question and your answer is just what I need !

Thank's for your help,
Brice.