streaming results

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

streaming results

Eduard Drenth
Dear all,

I am trying to stream the results of an xquery script, testing with

curl http://localhost:8080/exist/rest/db/apps/tdb/test.xq

But the script seems to first build the response and after that starts outputing.

Should I use response:stream? Do I need saxonica EE?

Eduard Drenth, Software Architekt

[hidden email]

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47

gpg: https://sks-keyservers.net/pks/lookup?op=get&search=0x065EF82A1E02CC43

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: streaming results

Wolfgang Meier-2
> But the script seems to first build the response and after that starts outputing.

Except for those calling response:streams explicitly, queries will always build the response before sending out results.

> Should I use response:stream? Do I need saxonica EE?

Saxon is not used for XQuery processing in eXist.

Wolfgang
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: streaming results

Eduard Drenth
ok...

I tested with this script:

    xquery version "3.1";

    declare namespace response = "http://exist-db.org/xquery/response";

    response:stream(
        for $s in //*[not(*)]
        return $s,
        ""
    )

works, but it doesn't start outputing any sooner compared to this:

     xquery version "3.1";

    declare namespace response = "http://exist-db.org/xquery/response";

    for $s in //*[not(*)]
    return $s

and when I do this, I only see the first element after a long time:

    xquery version "3.1";

    declare namespace response = "http://exist-db.org/xquery/response";

    for $s in //*[not(*)]
    return response:stream($s, "")

Is there a way to realy stream these results?

Eduard Drenth, Software Architekt

[hidden email]

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47

gpg: https://sks-keyservers.net/pks/lookup?op=get&search=0x065EF82A1E02CC43

________________________________________
From: Wolfgang Meier <[hidden email]>
Sent: Wednesday, June 7, 2017 3:25 PM
To: Eduard Drenth
Cc: [hidden email]
Subject: Re: [Exist-open] streaming results

> But the script seems to first build the response and after that starts outputing.

Except for those calling response:streams explicitly, queries will always build the response before sending out results.

> Should I use response:stream? Do I need saxonica EE?

Saxon is not used for XQuery processing in eXist.

Wolfgang

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: streaming results

Eduard Drenth
If I look here https://github.com/eXist-db/exist/blob/develop/src/org/exist/xquery/functions/response/Stream.java

it looks like Stream#eval can be / will be called only once with all output to be streamed.

It would be nice to be able to repeatedly call a function that streams for intermediary results. Perhaps not so simple though.

And perhaps I am mistaking....

Bye

Eduard Drenth, Software Architekt

[hidden email]

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47

gpg: https://sks-keyservers.net/pks/lookup?op=get&search=0x065EF82A1E02CC43

________________________________________
From: Eduard Drenth <[hidden email]>
Sent: Wednesday, June 7, 2017 4:36 PM
To: Wolfgang Meier
Cc: [hidden email]
Subject: Re: [Exist-open] streaming results

ok...

I tested with this script:

    xquery version "3.1";

    declare namespace response = "http://exist-db.org/xquery/response";

    response:stream(
        for $s in //*[not(*)]
        return $s,
        ""
    )

works, but it doesn't start outputing any sooner compared to this:

     xquery version "3.1";

    declare namespace response = "http://exist-db.org/xquery/response";

    for $s in //*[not(*)]
    return $s

and when I do this, I only see the first element after a long time:

    xquery version "3.1";

    declare namespace response = "http://exist-db.org/xquery/response";

    for $s in //*[not(*)]
    return response:stream($s, "")

Is there a way to realy stream these results?

Eduard Drenth, Software Architekt

[hidden email]

Doelestrjitte 8
8911 DX  Ljouwert
+31 58 234 30 47

gpg: https://sks-keyservers.net/pks/lookup?op=get&search=0x065EF82A1E02CC43

________________________________________
From: Wolfgang Meier <[hidden email]>
Sent: Wednesday, June 7, 2017 3:25 PM
To: Eduard Drenth
Cc: [hidden email]
Subject: Re: [Exist-open] streaming results

> But the script seems to first build the response and after that starts outputing.

Except for those calling response:streams explicitly, queries will always build the response before sending out results.

> Should I use response:stream? Do I need saxonica EE?

Saxon is not used for XQuery processing in eXist.

Wolfgang

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open