eXist coverage and XQuery F&O 3.1 CR

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

eXist coverage and XQuery F&O 3.1 CR

Joe Wicentowski
Hi all,

Last September I created a gist to analysis of 3.0RC1's coverage of
the functions listed in the XQuery 3.1 spec.  I updated the gist with
today's build of the develop branch:

  https://gist.github.com/joewiz/acb0558f078c0576a5e7

The areas of difference are now smaller: eXist has eliminated the
xsl:* functions and added support for map:merge(), map:put(), and
map:size().  Some differences remain, so I will list these below and
offer some commentary.  Of course, the 3.1 spec is not final, and as
recently as last week, Michael Kay announced a new function,
map:find().  Such changes make for a shifting target, which is
challenging for implementations to keep up with.  So, of course, my
purpose is not to criticize but to foster discussion about which areas
of difference are worth prioritizing, particularly as some of these
remaining functions aren't likely to change before the spec is
finalized.

## Functions that eXist still lacks

- array:sort
- fn:collation-key
- fn:contains-token
- fn:element-with-id
- fn:format-integer
- fn:has-children
- fn:innermost
- fn:load-xquery-module
- fn:outermost
- fn:parse-ietf-date
- fn:path
- fn:random-number-generator
- fn:sort
- fn:transform
- fn:unparsed-text
- fn:unparsed-text-available
- fn:unparsed-text-lines
- fn:uri-collection

## Discussion of functions eXist still lacks

Of these, I think the fn:unparsed-* functions are the nicest
additions, since they reduce lines of code for retrieving text files
from remote servers; for example, to retrieve a CSV file, we could
avoid using HTTP client, util:binary-to-string, and tokenize.

- format-integer() seems so within reach, since eXist already supports
format-number
- transform() might be within reach too, since eXist has transform:transform
- sort() and array:sort() would also reduce lines of code, a single
line replacing intermediate FLWORs
- other ideas?

## Caveats

I've limited myself to discussion of functions here (really, just
function names; I haven't gotten as deep as variants in function
signatures etc.).  I think everyone's aware of the lack of windowing,
the count clause, the arrow operator, and the string operator; or
things outside the core spec and F&O, like adaptive serialization or
the XQuery Update and XQuery Full Text facilities.

## Deprecating functions from previous revisions of the spec

The functions that eXist supports that have no basis in the spec are as follows:

- fn:equals
- fn:escape-uri
- fn:map
- fn:map-pairs
- map:map:for-each-entry
- map:new

Clearly some of these were attempts to support earlier versions of the
spec and have been left in for backwards compatibility - to allow code
written against earlier versions of the spec to continue working.
They're all candidates for deprecation.

Joe

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: eXist coverage and XQuery F&O 3.1 CR

Joe Wicentowski
> I think everyone's aware of the lack of windowing,
> the count clause, the arrow operator, and the string operator; or...

Sorry, I mistyped: I meant "string constructor", not "string operator."

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: eXist coverage and XQuery F&O 3.1 CR

Alister Pillow-3
In reply to this post by Joe Wicentowski
Hi Joe,

Thanks for producing this list of new functions.
+1 unparsed-text*
+1 format-integer
+1 parse-ietf-date

Anything to make handling dates easier - the bane of my life!

Regards,
Alister.


> On 13 Sep 2016, at 2:17 AM, Joe Wicentowski <[hidden email]> wrote:
>
> Hi all,
>
> Last September I created a gist to analysis of 3.0RC1's coverage of
> the functions listed in the XQuery 3.1 spec.  I updated the gist with
> today's build of the develop branch:
>
>  https://gist.github.com/joewiz/acb0558f078c0576a5e7
>
> The areas of difference are now smaller: eXist has eliminated the
> xsl:* functions and added support for map:merge(), map:put(), and
> map:size().  Some differences remain, so I will list these below and
> offer some commentary.  Of course, the 3.1 spec is not final, and as
> recently as last week, Michael Kay announced a new function,
> map:find().  Such changes make for a shifting target, which is
> challenging for implementations to keep up with.  So, of course, my
> purpose is not to criticize but to foster discussion about which areas
> of difference are worth prioritizing, particularly as some of these
> remaining functions aren't likely to change before the spec is
> finalized.
>
> ## Functions that eXist still lacks
>
> - array:sort
> - fn:collation-key
> - fn:contains-token
> - fn:element-with-id
> - fn:format-integer
> - fn:has-children
> - fn:innermost
> - fn:load-xquery-module
> - fn:outermost
> - fn:parse-ietf-date
> - fn:path
> - fn:random-number-generator
> - fn:sort
> - fn:transform
> - fn:unparsed-text
> - fn:unparsed-text-available
> - fn:unparsed-text-lines
> - fn:uri-collection
>
> ## Discussion of functions eXist still lacks
>
> Of these, I think the fn:unparsed-* functions are the nicest
> additions, since they reduce lines of code for retrieving text files
> from remote servers; for example, to retrieve a CSV file, we could
> avoid using HTTP client, util:binary-to-string, and tokenize.
>
> - format-integer() seems so within reach, since eXist already supports
> format-number
> - transform() might be within reach too, since eXist has transform:transform
> - sort() and array:sort() would also reduce lines of code, a single
> line replacing intermediate FLWORs
> - other ideas?
>
> ## Caveats
>
> I've limited myself to discussion of functions here (really, just
> function names; I haven't gotten as deep as variants in function
> signatures etc.).  I think everyone's aware of the lack of windowing,
> the count clause, the arrow operator, and the string operator; or
> things outside the core spec and F&O, like adaptive serialization or
> the XQuery Update and XQuery Full Text facilities.
>
> ## Deprecating functions from previous revisions of the spec
>
> The functions that eXist supports that have no basis in the spec are as follows:
>
> - fn:equals
> - fn:escape-uri
> - fn:map
> - fn:map-pairs
> - map:map:for-each-entry
> - map:new
>
> Clearly some of these were attempts to support earlier versions of the
> spec and have been left in for backwards compatibility - to allow code
> written against earlier versions of the spec to continue working.
> They're all candidates for deprecation.
>
> Joe
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. http://sdm.link/zohodev2dev
> _______________________________________________
> Exist-open mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/exist-open


------------------------------------------------------------------------------
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: eXist coverage and XQuery F&O 3.1 CR

Immanuel Normann
In reply to this post by Joe Wicentowski
Hi Joe,

I just wanted to use the map-module for the first time and now I am surprised that map:merge is not supported in existdb 2.2. I wonder how I can create maps dynamically with the map-functions supported in exist 2.2? The functions I see there is map:new (to create an empty map) and map:entry (for one-element-map).

What I'd like to have is something I found in the baseX docs (http://docs.basex.org/wiki/Map_Module):

map:merge(for $b in //book return map:entry($b/isbn, $b))

Is there a way to create maps dynamically in exist 2.2?

Regards,
Immanuel

2016-09-12 18:47 GMT+02:00 Joe Wicentowski <[hidden email]>:
Hi all,

Last September I created a gist to analysis of 3.0RC1's coverage of
the functions listed in the XQuery 3.1 spec.  I updated the gist with
today's build of the develop branch:

  https://gist.github.com/joewiz/acb0558f078c0576a5e7

The areas of difference are now smaller: eXist has eliminated the
xsl:* functions and added support for map:merge(), map:put(), and
map:size().  Some differences remain, so I will list these below and
offer some commentary.  Of course, the 3.1 spec is not final, and as
recently as last week, Michael Kay announced a new function,
map:find().  Such changes make for a shifting target, which is
challenging for implementations to keep up with.  So, of course, my
purpose is not to criticize but to foster discussion about which areas
of difference are worth prioritizing, particularly as some of these
remaining functions aren't likely to change before the spec is
finalized.

## Functions that eXist still lacks

- array:sort
- fn:collation-key
- fn:contains-token
- fn:element-with-id
- fn:format-integer
- fn:has-children
- fn:innermost
- fn:load-xquery-module
- fn:outermost
- fn:parse-ietf-date
- fn:path
- fn:random-number-generator
- fn:sort
- fn:transform
- fn:unparsed-text
- fn:unparsed-text-available
- fn:unparsed-text-lines
- fn:uri-collection

## Discussion of functions eXist still lacks

Of these, I think the fn:unparsed-* functions are the nicest
additions, since they reduce lines of code for retrieving text files
from remote servers; for example, to retrieve a CSV file, we could
avoid using HTTP client, util:binary-to-string, and tokenize.

- format-integer() seems so within reach, since eXist already supports
format-number
- transform() might be within reach too, since eXist has transform:transform
- sort() and array:sort() would also reduce lines of code, a single
line replacing intermediate FLWORs
- other ideas?

## Caveats

I've limited myself to discussion of functions here (really, just
function names; I haven't gotten as deep as variants in function
signatures etc.).  I think everyone's aware of the lack of windowing,
the count clause, the arrow operator, and the string operator; or
things outside the core spec and F&O, like adaptive serialization or
the XQuery Update and XQuery Full Text facilities.

## Deprecating functions from previous revisions of the spec

The functions that eXist supports that have no basis in the spec are as follows:

- fn:equals
- fn:escape-uri
- fn:map
- fn:map-pairs
- map:map:for-each-entry
- map:new

Clearly some of these were attempts to support earlier versions of the
spec and have been left in for backwards compatibility - to allow code
written against earlier versions of the spec to continue working.
They're all candidates for deprecation.

Joe

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity
planning reports. http://sdm.link/zohodev2dev
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: eXist coverage and XQuery F&O 3.1 CR

Wolfgang Meier-2
Hi Immanuel,

map:new is essentially the same as map:merge. The implementation in 2.2 was based on an older proposal for the map module. In 3.0 map:new got renamed to map:merge.

Wolfgang



> Am 11.10.2016 um 11:29 schrieb Immanuel Normann <[hidden email]>:
>
> Hi Joe,
>
> I just wanted to use the map-module for the first time and now I am surprised that map:merge is not supported in existdb 2.2. I wonder how I can create maps dynamically with the map-functions supported in exist 2.2? The functions I see there is map:new (to create an empty map) and map:entry (for one-element-map).
>
> What I'd like to have is something I found in the baseX docs (http://docs.basex.org/wiki/Map_Module):
>
> map:merge(for $b in //book return map:entry($b/isbn, $b))
>
> Is there a way to create maps dynamically in exist 2.2?
>
> Regards,
> Immanuel
>
> 2016-09-12 18:47 GMT+02:00 Joe Wicentowski <[hidden email]>:
> Hi all,
>
> Last September I created a gist to analysis of 3.0RC1's coverage of
> the functions listed in the XQuery 3.1 spec.  I updated the gist with
> today's build of the develop branch:
>
>   https://gist.github.com/joewiz/acb0558f078c0576a5e7
>
> The areas of difference are now smaller: eXist has eliminated the
> xsl:* functions and added support for map:merge(), map:put(), and
> map:size().  Some differences remain, so I will list these below and
> offer some commentary.  Of course, the 3.1 spec is not final, and as
> recently as last week, Michael Kay announced a new function,
> map:find().  Such changes make for a shifting target, which is
> challenging for implementations to keep up with.  So, of course, my
> purpose is not to criticize but to foster discussion about which areas
> of difference are worth prioritizing, particularly as some of these
> remaining functions aren't likely to change before the spec is
> finalized.
>
> ## Functions that eXist still lacks
>
> - array:sort
> - fn:collation-key
> - fn:contains-token
> - fn:element-with-id
> - fn:format-integer
> - fn:has-children
> - fn:innermost
> - fn:load-xquery-module
> - fn:outermost
> - fn:parse-ietf-date
> - fn:path
> - fn:random-number-generator
> - fn:sort
> - fn:transform
> - fn:unparsed-text
> - fn:unparsed-text-available
> - fn:unparsed-text-lines
> - fn:uri-collection
>
> ## Discussion of functions eXist still lacks
>
> Of these, I think the fn:unparsed-* functions are the nicest
> additions, since they reduce lines of code for retrieving text files
> from remote servers; for example, to retrieve a CSV file, we could
> avoid using HTTP client, util:binary-to-string, and tokenize.
>
> - format-integer() seems so within reach, since eXist already supports
> format-number
> - transform() might be within reach too, since eXist has transform:transform
> - sort() and array:sort() would also reduce lines of code, a single
> line replacing intermediate FLWORs
> - other ideas?
>
> ## Caveats
>
> I've limited myself to discussion of functions here (really, just
> function names; I haven't gotten as deep as variants in function
> signatures etc.).  I think everyone's aware of the lack of windowing,
> the count clause, the arrow operator, and the string operator; or
> things outside the core spec and F&O, like adaptive serialization or
> the XQuery Update and XQuery Full Text facilities.
>
> ## Deprecating functions from previous revisions of the spec
>
> The functions that eXist supports that have no basis in the spec are as follows:
>
> - fn:equals
> - fn:escape-uri
> - fn:map
> - fn:map-pairs
> - map:map:for-each-entry
> - map:new
>
> Clearly some of these were attempts to support earlier versions of the
> spec and have been left in for backwards compatibility - to allow code
> written against earlier versions of the spec to continue working.
> They're all candidates for deprecation.
>
> Joe
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. http://sdm.link/zohodev2dev
> _______________________________________________
> Exist-open mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot_______________________________________________
> Exist-open mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/exist-open


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: eXist coverage and XQuery F&O 3.1 CR

Joe Wicentowski
Hi all,

With last week's release of eXist 3.4.0, whose new support for fn:sort() makes it the latest in a string of releases adding functions and features from XQuery 3.1, I thought I would update my list of eXist's coverage of XQuery 3.1 functions—the first since the XQuery 3.1 achieved Recommendation status.  I've posted the raw data at https://gist.github.com/joewiz/acb0558f078c0576a5e7#gistcomment-2165426, so here is my analysis:

eXist now supports these functions not supported in my last list:

- fn:has-children
- fn:innermost
- fn:load-xquery-module
- fn:sort
- (eXist also supports the arrow operator)

Since the last list the W3C added these functions, which eXist has not yet implemented:

- array:put
- fn:default-language
- fn:json-to-xml
- fn:xml-to-json

In total, therefore, eXist now lacks support for only the following 18 functions:

- array:put
- array:sort
- fn:collation-key
- fn:contains-token
- fn:default-language
- fn:element-with-id
- fn:format-integer
- fn:json-to-xml
- fn:outermost
- fn:parse-ietf-date
- fn:path
- fn:random-number-generator
- fn:transform
- fn:unparsed-text
- fn:unparsed-text-available
- fn:unparsed-text-lines
- fn:uri-collection
- fn:xml-to-json

(Of these, my top votes go to: unparsed-text*, format-integer.  Alister voiced support for these as well as parse-ietf-date.)

eXist still has some functions in the standard namespaces that are not in the spec (based on previous drafts, I believe)—easy candidates for deprecation:

- fn:equals
- fn:escape-uri
- fn:map
- fn:map-pairs

(For anyone who needs support for the functionality in fn:xml-to-json and fn:json-to-xml, I have implemented these in XQuery. See https://gist.github.com/joewiz/d986da715facaad633db.)

Joe


---------- Forwarded message ----------
From: Alister Pillow <[hidden email]>
Date: Tue, Sep 13, 2016 at 7:02 PM
Subject: Re: [Exist-open] eXist coverage and XQuery F&O 3.1 CR
To: Joe Wicentowski <[hidden email]>
Cc: exist-open <[hidden email]>


Hi Joe,

Thanks for producing this list of new functions.
+1 unparsed-text*
+1 format-integer
+1 parse-ietf-date

Anything to make handling dates easier - the bane of my life!

Regards,
Alister.


On Tue, Oct 11, 2016 at 5:35 AM, Wolfgang Meier <[hidden email]> wrote:
Hi Immanuel,

map:new is essentially the same as map:merge. The implementation in 2.2 was based on an older proposal for the map module. In 3.0 map:new got renamed to map:merge.

Wolfgang



> Am 11.10.2016 um 11:29 schrieb Immanuel Normann <[hidden email]>:
>
> Hi Joe,
>
> I just wanted to use the map-module for the first time and now I am surprised that map:merge is not supported in existdb 2.2. I wonder how I can create maps dynamically with the map-functions supported in exist 2.2? The functions I see there is map:new (to create an empty map) and map:entry (for one-element-map).
>
> What I'd like to have is something I found in the baseX docs (http://docs.basex.org/wiki/Map_Module):
>
> map:merge(for $b in //book return map:entry($b/isbn, $b))
>
> Is there a way to create maps dynamically in exist 2.2?
>
> Regards,
> Immanuel
>
> 2016-09-12 18:47 GMT+02:00 Joe Wicentowski <[hidden email]>:
> Hi all,
>
> Last September I created a gist to analysis of 3.0RC1's coverage of
> the functions listed in the XQuery 3.1 spec.  I updated the gist with
> today's build of the develop branch:
>
>   https://gist.github.com/joewiz/acb0558f078c0576a5e7
>
> The areas of difference are now smaller: eXist has eliminated the
> xsl:* functions and added support for map:merge(), map:put(), and
> map:size().  Some differences remain, so I will list these below and
> offer some commentary.  Of course, the 3.1 spec is not final, and as
> recently as last week, Michael Kay announced a new function,
> map:find().  Such changes make for a shifting target, which is
> challenging for implementations to keep up with.  So, of course, my
> purpose is not to criticize but to foster discussion about which areas
> of difference are worth prioritizing, particularly as some of these
> remaining functions aren't likely to change before the spec is
> finalized.
>
> ## Functions that eXist still lacks
>
> - array:sort
> - fn:collation-key
> - fn:contains-token
> - fn:element-with-id
> - fn:format-integer
> - fn:has-children
> - fn:innermost
> - fn:load-xquery-module
> - fn:outermost
> - fn:parse-ietf-date
> - fn:path
> - fn:random-number-generator
> - fn:sort
> - fn:transform
> - fn:unparsed-text
> - fn:unparsed-text-available
> - fn:unparsed-text-lines
> - fn:uri-collection
>
> ## Discussion of functions eXist still lacks
>
> Of these, I think the fn:unparsed-* functions are the nicest
> additions, since they reduce lines of code for retrieving text files
> from remote servers; for example, to retrieve a CSV file, we could
> avoid using HTTP client, util:binary-to-string, and tokenize.
>
> - format-integer() seems so within reach, since eXist already supports
> format-number
> - transform() might be within reach too, since eXist has transform:transform
> - sort() and array:sort() would also reduce lines of code, a single
> line replacing intermediate FLWORs
> - other ideas?
>
> ## Caveats
>
> I've limited myself to discussion of functions here (really, just
> function names; I haven't gotten as deep as variants in function
> signatures etc.).  I think everyone's aware of the lack of windowing,
> the count clause, the arrow operator, and the string operator; or
> things outside the core spec and F&O, like adaptive serialization or
> the XQuery Update and XQuery Full Text facilities.
>
> ## Deprecating functions from previous revisions of the spec
>
> The functions that eXist supports that have no basis in the spec are as follows:
>
> - fn:equals
> - fn:escape-uri
> - fn:map
> - fn:map-pairs
> - map:map:for-each-entry
> - map:new
>
> Clearly some of these were attempts to support earlier versions of the
> spec and have been left in for backwards compatibility - to allow code
> written against earlier versions of the spec to continue working.
> They're all candidates for deprecation.
>
> Joe
>
> ------------------------------------------------------------------------------
> What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
> patterns at an interface-level. Reveals which users, apps, and protocols are
> consuming the most bandwidth. Provides multi-vendor support for NetFlow,
> J-Flow, sFlow and other flows. Make informed decisions using capacity
> planning reports. http://sdm.link/zohodev2dev
> _______________________________________________
> Exist-open mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/exist-open
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot_______________________________________________


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Loading...