Quantcast

Test Suite Error Reporting

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Test Suite Error Reporting

Jonathan Rowell

Hi,


using the latest eXist (see below) I'm running some tests using the Test Suite interface via Grunt (I get the same results in eXide) and I get the following error :-


<testcase name="do-media-labelclass="inptest:do-media-label">

<error type="err:XPTY0004message="It is a type error if, during the static analysis phase, an expression is found to have a static type that is not appropriate for the context in which the expression occurs, or during the dynamic evaluation phase, the dynamic type of a value does not match a required type as specified by the matching rules in 2.5.4 SequenceType Matching. xs:integer(9) is not a sub-type of xs:string"/>
</testcase>



Obviously the test suite mechanism has caught the exception (on calling test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it into a message - all very nice, but where is the error? The function inptest:do-media-label is of course a one liner which simply calls a function in a library module, which is what I am actually testing. It would be wonderful to be told where the error is.


And a minor point is in the header :-


<testsuite package="http://www.JonathanTest.fr/tests/input-checktimestamp="2017-02-25T15:38:01.528+01:00failures="0pending="0tests="3"


that there are indeed three tests all of which receive the same error, yet the attribute "failures" is zero.


regards


Jonathan


And now to find out where the type error comes from .........................



#eXist build info
project.version=3.0
project.built=20170208065653
scm.branch=develop
scm.revision=acd0c14



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Test Suite Error Reporting

Dmitriy Shabanov
Do you have access to console output?

On Sat, Feb 25, 2017 at 6:06 PM, Jonathan Rowell <[hidden email]> wrote:

Hi,


using the latest eXist (see below) I'm running some tests using the Test Suite interface via Grunt (I get the same results in eXide) and I get the following error :-


<testcase name="do-media-labelclass="inptest:do-media-label">

<error type="err:XPTY0004message="It is a type error if, during the static analysis phase, an expression is found to have a static type that is not appropriate for the context in which the expression occurs, or during the dynamic evaluation phase, the dynamic type of a value does not match a required type as specified by the matching rules in 2.5.4 SequenceType Matching. xs:integer(9) is not a sub-type of xs:string"/>
</testcase>



Obviously the test suite mechanism has caught the exception (on calling test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it into a message - all very nice, but where is the error? The function inptest:do-media-label is of course a one liner which simply calls a function in a library module, which is what I am actually testing. It would be wonderful to be told where the error is.


And a minor point is in the header :-


<testsuite package="http://www.JonathanTest.fr/tests/input-checktimestamp="2017-02-25T15:38:01.528+01:00failures="0pending="0tests="3"


that there are indeed three tests all of which receive the same error, yet the attribute "failures" is zero.


regards


Jonathan


And now to find out where the type error comes from .........................



#eXist build info
project.version=3.0
project.built=20170208065653
scm.branch=develop
scm.revision=acd0c14



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open




--
Dmitriy Shabanov

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Test Suite Error Reporting

Jonathan Rowell

Hi Dmitriy,


I normally run eXist as a service on Windows. I presume I could start it in a DOS box via a script. 


Although that might be a way to find my error, I'm much more interested in a complete message for the general user. I have already completed a Grunt unloader (remove package), Installer and Package checker - which tests all the XQueries for compilation and that the .xconf files are correctly loaded, and this Grunt utility is to run the test suites. So far I've not been able to complete these and relase them due to errors and problems in 3.0.


But I'm ready for any ideas you might have


regards


Jonathan




From: Dmitriy Shabanov <[hidden email]>
Sent: Saturday, February 25, 2017 3:16 PM
To: Jonathan Rowell
Cc: eXist open
Subject: Re: [Exist-open] Test Suite Error Reporting
 
Do you have access to console output?

On Sat, Feb 25, 2017 at 6:06 PM, Jonathan Rowell <[hidden email]> wrote:

Hi,


using the latest eXist (see below) I'm running some tests using the Test Suite interface via Grunt (I get the same results in eXide) and I get the following error :-


<testcase name="do-media-labelclass="inptest:do-media-label">

<error type="err:XPTY0004message="It is a type error if, during the static analysis phase, an expression is found to have a static type that is not appropriate for the context in which the expression occurs, or during the dynamic evaluation phase, the dynamic type of a value does not match a required type as specified by the matching rules in 2.5.4 SequenceType Matching. xs:integer(9) is not a sub-type of xs:string"/>
</testcase>



Obviously the test suite mechanism has caught the exception (on calling test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it into a message - all very nice, but where is the error? The function inptest:do-media-label is of course a one liner which simply calls a function in a library module, which is what I am actually testing. It would be wonderful to be told where the error is.


And a minor point is in the header :-


<testsuite package="http://www.JonathanTest.fr/tests/input-checktimestamp="2017-02-25T15:38:01.528+01:00failures="0pending="0tests="3"


that there are indeed three tests all of which receive the same error, yet the attribute "failures" is zero.


regards


Jonathan


And now to find out where the type error comes from .........................



#eXist build info
project.version=3.0
project.built=20170208065653
scm.branch=develop
scm.revision=acd0c14



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open




--
Dmitriy Shabanov

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Test Suite Error Reporting

Grant MacKenzie
In reply to this post by Jonathan Rowell
I agree here with Jonathan

I think any test or compile error/warning  should generate a
-  an  error code and description/message
-  output of the test  ..  so you can compare what is expected with what you got
- file  path/name
- line number
- optional col number

Without the file name and line number., tests become a bit of a
scratch the head exercise.  The baseX test output  will always have
<error line="55" column="6" type="FORG0001">
I think eXist should aim for the same.

Before I run any tests I always compile first ....  code - compile -
upload- test.

What I think you have got is a compile error when the tests run,  and
that is why you get
failures="0" pending="0" tests="3

If you ask eXist to compile using the rest API
 util:compile-query($expression as xs:string, $module-load-path as
xs:string?) as
element()
or util:compile($expression as xs:string, $module-load-path as xs:string) as xs
:string

prior to uploading code an running the tests then I think you would
get a better code-test cycle

One thing  I think that would be handy, would be a compile 'stops at '
line number,

>    1. Test Suite Error Reporting (Jy Rowell)
>    2. Re: Test Suite Error Reporting (Dmitriy Shabanov)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 25 Feb 2017 15:06:10 +0000
> From: Jonathan Rowell <[hidden email]>
> Subject: [Exist-open] Test Suite Error Reporting
> To: eXist open <[hidden email]>
> Message-ID:
>         <[hidden email]>
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
>
> using the latest eXist (see below) I'm running some tests using the Test Suite interface via Grunt (I get the same results in eXide) and I get the following error :-
>
>
> <testcase name="do-media-label" class="inptest:do-media-label">
>
> <error type="err:XPTY0004" message="It is a type error if, during the static analysis phase, an expression is found to have a static type that is not appropriate for the context in which the expression occurs, or during the dynamic evaluation phase, the dynamic type of a value does not match a required type as specified by the matching rules in 2.5.4 SequenceType Matching. xs:integer(9) is not a sub-type of xs:string"/>
> </testcase>
>
>
>
> Obviously the test suite mechanism has caught the exception (on calling test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it into a message - all very nice, but where is the error? The function inptest:do-media-label is of course a one liner which simply calls a function in a library module, which is what I am actually testing. It would be wonderful to be told where the error is.
>
>
> And a minor point is in the header :-
>
>
> <testsuite package="http://www.JonathanTest.fr/tests/input-check" timestamp="2017-02-25T15:38:01.528+01:00" failures="0" pending="0" tests="3"
>
>
> that there are indeed three tests all of which receive the same error, yet the attribute "failures" is zero.
>
>
> regards
>
>
> Jonathan
>
>
> And now to find out where the type error comes from .........................
>
>
>
> #eXist build info
> project.version=3.0
> project.built=20170208065653
> scm.branch=develop
> scm.revision=acd0c14
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
>
> ------------------------------
>
> Message: 2
> Date: Sat, 25 Feb 2017 18:16:46 +0300
> From: Dmitriy Shabanov <[hidden email]>
> Subject: Re: [Exist-open] Test Suite Error Reporting
> To: Jonathan Rowell <[hidden email]>
> Cc: eXist open <[hidden email]>
> Message-ID:
>         <CADD4p=[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
> Do you have access to console output?
>
> On Sat, Feb 25, 2017 at 6:06 PM, Jonathan Rowell <[hidden email]>
> wrote:
>
>> Hi,
>>
>>
>> using the latest eXist (see below) I'm running some tests using the Test
>> Suite interface via Grunt (I get the same results in eXide) and I get the
>> following error :-
>>
>>
>> <testcase name="do-media-label" class="inptest:do-media-label">
>> <error type="err:XPTY0004" message="It is a type error if, during the
>> static analysis phase, an expression is found to have a static type that is
>> not appropriate for the context in which the expression occurs, or during
>> the dynamic evaluation phase, the dynamic type of a value does not match a
>> required type as specified by the matching rules in 2.5.4 SequenceType
>> Matching. xs:integer(9) is not a sub-type of xs:string"/>
>> </testcase>
>>
>>
>>
>> Obviously the test suite mechanism has caught the exception (on calling
>> test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it
>> into a message - all very nice, but where is the error? The function
>> inptest:do-media-label is of course a one liner which simply calls a
>> function in a library module, which is what I am actually testing. It would
>> be wonderful to be told where the error is.
>>
>>
>> And a minor point is in the header :-
>>
>>
>> <testsuite package="http://www.JonathanTest.fr/tests/input-check"
>> timestamp="2017-02-25T15:38:01.528+01:00" failures="0" pending="0" tests="
>> 3"
>>
>>
>> that there are indeed three tests all of which receive the same error, yet
>> the attribute "failures" is zero.
>>
>>
>> regards
>>
>>
>> Jonathan
>>
>>
>> And now to find out where the type error comes from
>> .........................
>>
>>
>>
>> #eXist build info
>> project.version=3.0
>> project.built=20170208065653
>> scm.branch=develop
>> scm.revision=acd0c14
>>
>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Exist-open mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/exist-open
>>
>>
>
>
> --
> Dmitriy Shabanov
> -------------- next part --------------
> An HTML attachment was scrubbed...
>
> ------------------------------
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
>
> ------------------------------
>
> _______________________________________________
> Exist-open mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
>
> End of Exist-open Digest, Vol 130, Issue 70
> *******************************************



--
Take Care
Grant Mackenzie

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Test Suite Error Reporting

Jonathan Rowell

Thanks for your support, Grant.


Finding the error was not terribly difficult but would have been easier with a file and line number.


I think I have seen an exception raised during a test suite with an error on StackOverflow, but for the life of me I can't find it again. It showed line numbers and a file name. I believe it was with eXist 2.2.

I have started to scan the source looking for this error. Anybody got any pointers on where I should look?


This link shows the case in question, although mine was at run time. 

http://stackoverflow.com/questions/36010788/how-to-test-complex-functions-in-exist-db-with-xqsuite/36017116

and this link shows up the other problem 

Here when a failure occurs the count gets incremented. I don't rely on that, I simply look for error or failure tags.

regards to all

Jonathan




From: Grant MacKenzie <[hidden email]>
Sent: Saturday, February 25, 2017 9:01 PM
To: [hidden email]
Subject: Re: [Exist-open] Test Suite Error Reporting
 
I agree here with Jonathan

I think any test or compile error/warning  should generate a
-  an  error code and description/message
-  output of the test  ..  so you can compare what is expected with what you got
- file  path/name
- line number
- optional col number

Without the file name and line number., tests become a bit of a
scratch the head exercise.  The baseX test output  will always have
<error line="55" column="6" type="FORG0001">
I think eXist should aim for the same.

Before I run any tests I always compile first ....  code - compile -
upload- test.

What I think you have got is a compile error when the tests run,  and
that is why you get
failures="0" pending="0" tests="3

If you ask eXist to compile using the rest API
 util:compile-query($expression as xs:string, $module-load-path as
xs:string?) as
element()
or util:compile($expression as xs:string, $module-load-path as xs:string) as xs
:string

prior to uploading code an running the tests then I think you would
get a better code-test cycle

One thing  I think that would be handy, would be a compile 'stops at '
line number,

>    1. Test Suite Error Reporting (Jy Rowell)
>    2. Re: Test Suite Error Reporting (Dmitriy Shabanov)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 25 Feb 2017 15:06:10 +0000
> From: Jonathan Rowell <[hidden email]>
> Subject: [Exist-open] Test Suite Error Reporting
> To: eXist open <[hidden email]>
> Message-ID:
>         <[hidden email]>
>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
>
> using the latest eXist (see below) I'm running some tests using the Test Suite interface via Grunt (I get the same results in eXide) and I get the following error :-
>
>
> <testcase name="do-media-label" class="inptest:do-media-label">
>
> <error type="err:XPTY0004" message="It is a type error if, during the static analysis phase, an expression is found to have a static type that is not appropriate for the context in which the expression occurs, or during the dynamic evaluation phase, the dynamic type of a value does not match a required type as specified by the matching rules in 2.5.4 SequenceType Matching. xs:integer(9) is not a sub-type of xs:string"/>
> </testcase>
>
>
>
> Obviously the test suite mechanism has caught the exception (on calling test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it into a message - all very nice, but where is the error? The function inptest:do-media-label is of course a one liner which simply calls a function in a library module, which is what I am actually testing. It would be wonderful to be told where the error is.
>
>
> And a minor point is in the header :-
>
>
> <testsuite package="http://www.JonathanTest.fr/tests/input-check" timestamp="2017-02-25T15:38:01.528+01:00" failures="0" pending="0" tests="3"
>
>
> that there are indeed three tests all of which receive the same error, yet the attribute "failures" is zero.
>
>
> regards
>
>
> Jonathan
>
>
> And now to find out where the type error comes from .........................
>
>
>
> #eXist build info
> project.version=3.0
> project.built=20170208065653
> scm.branch=develop
> scm.revision=acd0c14
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
>
> ------------------------------
>
> Message: 2
> Date: Sat, 25 Feb 2017 18:16:46 +0300
> From: Dmitriy Shabanov <[hidden email]>
> Subject: Re: [Exist-open] Test Suite Error Reporting
> To: Jonathan Rowell <[hidden email]>
> Cc: eXist open <[hidden email]>
> Message-ID:
>         <CADD4p=[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
> Do you have access to console output?
>
> On Sat, Feb 25, 2017 at 6:06 PM, Jonathan Rowell <[hidden email]>
> wrote:
>
>> Hi,
>>
>>
>> using the latest eXist (see below) I'm running some tests using the Test
>> Suite interface via Grunt (I get the same results in eXide) and I get the
>> following error :-
>>
>>
>> <testcase name="do-media-label" class="inptest:do-media-label">
>> <error type="err:XPTY0004" message="It is a type error if, during the
>> static analysis phase, an expression is found to have a static type that is
>> not appropriate for the context in which the expression occurs, or during
>> the dynamic evaluation phase, the dynamic type of a value does not match a
>> required type as specified by the matching rules in 2.5.4 SequenceType
>> Matching. xs:integer(9) is not a sub-type of xs:string"/>
>> </testcase>
>>
>>
>>
>> Obviously the test suite mechanism has caught the exception (on calling
>> test:suite(inspect:module-functions(xs:anyURI($mod-name))) and turned it
>> into a message - all very nice, but where is the error? The function
>> inptest:do-media-label is of course a one liner which simply calls a
>> function in a library module, which is what I am actually testing. It would
>> be wonderful to be told where the error is.
>>
>>
>> And a minor point is in the header :-
>>
>>
>> <testsuite package="http://www.JonathanTest.fr/tests/input-check"
>> timestamp="2017-02-25T15:38:01.528+01:00" failures="0" pending="0" tests="
>> 3"
>>
>>
>> that there are indeed three tests all of which receive the same error, yet
>> the attribute "failures" is zero.
>>
>>
>> regards
>>
>>
>> Jonathan
>>
>>
>> And now to find out where the type error comes from
>> .........................
>>
>>
>>
>> #eXist build info
>> project.version=3.0
>> project.built=20170208065653
>> scm.branch=develop
>> scm.revision=acd0c14
>>
>>
>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
sdm.link
Slashdot: News for nerds, stuff that matters. Timely news source for technology related news with a heavy slant towards Linux and Open Source issues.


>> _______________________________________________
>> Exist-open mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/exist-open
lists.sourceforge.net
Your email address: Your name (optional): You may enter a privacy password below. This provides only mild security, but should prevent others from messing with ...


>>
>>
>
>
> --
> Dmitriy Shabanov
> -------------- next part --------------
> An HTML attachment was scrubbed...
>
> ------------------------------
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
sdm.link
Slashdot: News for nerds, stuff that matters. Timely news source for technology related news with a heavy slant towards Linux and Open Source issues.


>
> ------------------------------
>
> _______________________________________________
> Exist-open mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/exist-open
lists.sourceforge.net
Your email address: Your name (optional): You may enter a privacy password below. This provides only mild security, but should prevent others from messing with ...


>
>
> End of Exist-open Digest, Vol 130, Issue 70
> *******************************************



--
Take Care
Grant Mackenzie

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
sdm.link
Slashdot: News for nerds, stuff that matters. Timely news source for technology related news with a heavy slant towards Linux and Open Source issues.


_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
lists.sourceforge.net
Your email address: Your name (optional): You may enter a privacy password below. This provides only mild security, but should prevent others from messing with ...



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Test Suite Error Reporting

Dmitriy Shabanov


On Sun, Feb 26, 2017 at 9:51 PM, Jonathan Rowell <[hidden email]> wrote:

Thanks for your support, Grant.


Finding the error was not terribly difficult but would have been easier with a file and line number.


I think that this change should address the issue. The problem was that module-functions return empty sequence when provided xquery script had any errors.

--
Dmitriy Shabanov

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Loading...