exist as service and reverse proxying

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

exist as service and reverse proxying

Pietro Liuzzo
Dear all,

I was trying to proxy my app on an Ubuntu server with Apache 2. 


although the .conf file is located somewhere else.

I tried changing several things but I get always 

You don't have permission to access /foo on this server.
Apache/2.4.7 (Ubuntu) Server at ..... Port 80

I have never done this sort of thing before, but I expected that given the collection name provided in the .conf file I could than direct my browser to

www.example.de/collection/ and get to my app. 

I can currently reach it from servername:8080/exist/apps/foo 

Might it be that this is due to the fact that I am running exist as a service as described here: http://exist-db.org/exist/apps/doc/advanced-installation.xml?q=install%20headless&field=all&id=D2.2.3#D2.2.3
?

is there perhaps a better way to proxy exist as recommended?
 
Thanks a lot for your help
Pietro
--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Peter Stadler
Hmm, I don’t think there is a connection between the wrapper and the apache proxy. I’m running the same setup and it works – I only had some trouble migrating my apache conf files from apache version 2.2 to 2.4.
What I learned: Order is important!! directories and aliases will be overwritten by location directives (see https://httpd.apache.org/docs/2.4/sections.html#merging
And maybe you’re facing some security restrictions? Have you tried to explicitly set "Require all granted“ (just for testing)?

Hope that helps
Best
Peter

Am 06.07.2016 um 11:37 schrieb Pietro Liuzzo <[hidden email]>:

Dear all,

I was trying to proxy my app on an Ubuntu server with Apache 2. 


although the .conf file is located somewhere else.

I tried changing several things but I get always 

You don't have permission to access /foo on this server.
Apache/2.4.7 (Ubuntu) Server at ..... Port 80

I have never done this sort of thing before, but I expected that given the collection name provided in the .conf file I could than direct my browser to

www.example.de/collection/ and get to my app. 

I can currently reach it from servername:8080/exist/apps/foo 

Might it be that this is due to the fact that I am running exist as a service as described here: http://exist-db.org/exist/apps/doc/advanced-installation.xml?q=install%20headless&field=all&id=D2.2.3#D2.2.3
?

is there perhaps a better way to proxy exist as recommended?
 
Thanks a lot for your help
Pietro
--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open

signature.asc (465 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Pietro Liuzzo
Dear all,

On the same topic,I tried also with Nginx to see if this was working there but without results.

I have copied the example, changed the server name to the name of my host server, the proxy_pass to the private URI of my app but I only get a Not Found response...

anyone has ideas where I could have made a mistake? Thanks a lot!
Pietro


On 6 July 2016 at 21:17, Peter Stadler <[hidden email]> wrote:
Hmm, I don’t think there is a connection between the wrapper and the apache proxy. I’m running the same setup and it works – I only had some trouble migrating my apache conf files from apache version 2.2 to 2.4.
What I learned: Order is important!! directories and aliases will be overwritten by location directives (see https://httpd.apache.org/docs/2.4/sections.html#merging
And maybe you’re facing some security restrictions? Have you tried to explicitly set "Require all granted“ (just for testing)?

Hope that helps
Best
Peter

Am 06.07.2016 um 11:37 schrieb Pietro Liuzzo <[hidden email]>:

Dear all,

I was trying to proxy my app on an Ubuntu server with Apache 2. 


although the .conf file is located somewhere else.

I tried changing several things but I get always 

You don't have permission to access /foo on this server.
Apache/2.4.7 (Ubuntu) Server at ..... Port 80

I have never done this sort of thing before, but I expected that given the collection name provided in the .conf file I could than direct my browser to

www.example.de/collection/ and get to my app. 

I can currently reach it from servername:8080/exist/apps/foo 

Might it be that this is due to the fact that I am running exist as a service as described here: http://exist-db.org/exist/apps/doc/advanced-installation.xml?q=install%20headless&field=all&id=D2.2.3#D2.2.3
?

is there perhaps a better way to proxy exist as recommended?
 
Thanks a lot for your help
Pietro
--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open




--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Joe Wicentowski
Hi Pietro,

You might check nginx's logs for more info about the failed requests. 

For a working example of an nginx configuration, see https://github.com/HistoryAtState/hsg-project/blob/master/deploy/1861/etc/nginx/vhosts/1861.hsg.conf.

Joe

Sent from my iPhone




On Mon, Jul 11, 2016 at 7:20 AM -0400, "Pietro Liuzzo" <[hidden email]> wrote:

Dear all,

On the same topic,I tried also with Nginx to see if this was working there but without results.

I have copied the example, changed the server name to the name of my host server, the proxy_pass to the private URI of my app but I only get a Not Found response...

anyone has ideas where I could have made a mistake? Thanks a lot!
Pietro


On 6 July 2016 at 21:17, Peter Stadler <[hidden email]> wrote:
Hmm, I don’t think there is a connection between the wrapper and the apache proxy. I’m running the same setup and it works – I only had some trouble migrating my apache conf files from apache version 2.2 to 2.4.
What I learned: Order is important!! directories and aliases will be overwritten by location directives (see https://httpd.apache.org/docs/2.4/sections.html#merging
And maybe you’re facing some security restrictions? Have you tried to explicitly set "Require all granted“ (just for testing)?

Hope that helps
Best
Peter

Am 06.07.2016 um 11:37 schrieb Pietro Liuzzo <[hidden email]>:

Dear all,

I was trying to proxy my app on an Ubuntu server with Apache 2. 


although the .conf file is located somewhere else.

I tried changing several things but I get always 

You don't have permission to access /foo on this server.
Apache/2.4.7 (Ubuntu) Server at ..... Port 80

I have never done this sort of thing before, but I expected that given the collection name provided in the .conf file I could than direct my browser to

www.example.de/collection/ and get to my app. 

I can currently reach it from servername:8080/exist/apps/foo 

Might it be that this is due to the fact that I am running exist as a service as described here: http://exist-db.org/exist/apps/doc/advanced-installation.xml?q=install%20headless&field=all&id=D2.2.3#D2.2.3
?

is there perhaps a better way to proxy exist as recommended?
 
Thanks a lot for your help
Pietro
--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open




--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Pietro Liuzzo
Dear Joe and dear all, 

this was very helpful again, thanks a lot!

I am now able to reach the rest and see my app that way, but still for some reason, with the very same configuration for location I cannot reach the actual app. 

The two options I wanted look to me exactly the same, but for some reason which I do not understand, while the second returns the xml view of the rest server, requesting the first returns a 404 as the server searches for the file in /usr/share/nginx/html/exist/apps/myapp/

If I use the same path as is I do get to the app as wanted and it runs as expected.

 
Here is the snippet from nginx.conf

 location /A/ {

                proxy_pass              http://localhost:8080/exist/apps/myapp/;

                proxy_set_header        Host            $host;

                proxy_set_header        X-Real-IP       $remote_addr;

                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

                proxy_set_header        nginx-request-uri $request_uri;

                proxy_pass_header       Set-Cookie;

                proxy_cache             off;

                    }


        location /B/ {

                proxy_pass              http://localhost:8080/exist/rest/db/apps/myapp/;

                proxy_set_header        X-Real-IP       $remote_addr;

                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

                proxy_set_header        nginx-request-uri $request_uri;

                proxy_pass_header       Set-Cookie;

                proxy_cache             off;

                proxy_set_header        Host            $host;


                }

changing the path in A to exist/db/apps/myapp/ or /db/apps/myapp/ as in the examples of the book and of the website documentation 
also returns a 404, but not from Nginx, from Jetty.

I have also tried updating controller-config.xml adding a  <root/> line as I tried to do for apache in my earlier attempt and to change the redirecting instructions in the controller.xql within my app, but none of these attempts worked until now.

Thanks for your patience and for any further help!

sincerely yours

Pietro


On 11 July 2016 at 13:42, Joe Wicentowski <[hidden email]> wrote:
Hi Pietro,

You might check nginx's logs for more info about the failed requests. 


Joe

Sent from my iPhone




On Mon, Jul 11, 2016 at 7:20 AM -0400, "Pietro Liuzzo" <[hidden email]> wrote:

Dear all,

On the same topic,I tried also with Nginx to see if this was working there but without results.

I have copied the example, changed the server name to the name of my host server, the proxy_pass to the private URI of my app but I only get a Not Found response...

anyone has ideas where I could have made a mistake? Thanks a lot!
Pietro


On 6 July 2016 at 21:17, Peter Stadler <[hidden email]> wrote:
Hmm, I don’t think there is a connection between the wrapper and the apache proxy. I’m running the same setup and it works – I only had some trouble migrating my apache conf files from apache version 2.2 to 2.4.
What I learned: Order is important!! directories and aliases will be overwritten by location directives (see https://httpd.apache.org/docs/2.4/sections.html#merging
And maybe you’re facing some security restrictions? Have you tried to explicitly set "Require all granted“ (just for testing)?

Hope that helps
Best
Peter

Am 06.07.2016 um 11:37 schrieb Pietro Liuzzo <[hidden email]>:

Dear all,

I was trying to proxy my app on an Ubuntu server with Apache 2. 


although the .conf file is located somewhere else.

I tried changing several things but I get always 

You don't have permission to access /foo on this server.
Apache/2.4.7 (Ubuntu) Server at ..... Port 80

I have never done this sort of thing before, but I expected that given the collection name provided in the .conf file I could than direct my browser to

www.example.de/collection/ and get to my app. 

I can currently reach it from servername:8080/exist/apps/foo 

Might it be that this is due to the fact that I am running exist as a service as described here: http://exist-db.org/exist/apps/doc/advanced-installation.xml?q=install%20headless&field=all&id=D2.2.3#D2.2.3
?

is there perhaps a better way to proxy exist as recommended?
 
Thanks a lot for your help
Pietro
--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)
------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open




--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)



--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Joe Wicentowski
Hi Pietro,

I'd expect your '/A/' example to work, so I would suggest (1) running
`nginx -t` to test your nginx.conf configuration file for errors
preventing nginx from successfully picking up your changes to
nginx.conf and (2) scrutinize nginx's access and error logs (you've
found these, right?) and Jetty's request logs
($EXIST_HOME/tools/jetty/logs).  The idea is to ensure that nginx is
making the proxied request to the expected path for eXist.  (3) A
great troubleshooting tool is to log various request- and
context-related variables in in your controller.xql, using util:log or
console:log.  Good candidates for logging include: all of the $exist:*
variables, request:get-uri(), and
request:get-parameter('nginx-request-uri').  For example, see this use
of console:log in a controller.xql file:
https://github.com/eXistSolutions/hsg-shell/blob/master/controller.xql#L26-L33.
Once you're sure that the requests are getting through to eXist at the
configured nginx & Jetty URLs, you may need to adjust aspects of your
eXist application that expect the `/exist/apps/myapp` URL context so
that links resolve correctly, whether you're hitting eXist directly or
via nginx.  For example, see
https://github.com/eXistSolutions/hsg-shell/blob/master/modules/app.xqm#L9-L21.

Joe

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Joe Wicentowski
Hi Pietro,

Good, it sounds like you're getting close.  Check the redirects
leading to "index.html" in controller.xql.  My guess is that a leading
or trailing slash is getting added or removed in the redirect.

Joe

On Tue, Jul 12, 2016 at 6:53 AM, Pietro Liuzzo <[hidden email]> wrote:

> I have gone down to (2) for now and I think the path expected from Jetty is
> correct as it looks in the nginx.conf, but gets to Jetty from the log says
> doubled:
> exist/apps/myapp/
> arrives at Jetty as
> /exist/apps/myapp/exist/apps/myapp/index.html
>
> which clearly cannot be found.
>
> I think this happens within exist, but I am not sure where...
>
> Thanks for the further help!
>
>
>
>
> On 11 July 2016 at 20:03, Joe Wicentowski <[hidden email]> wrote:
>>
>> Hi Pietro,
>>
>> I'd expect your '/A/' example to work, so I would suggest (1) running
>> `nginx -t` to test your nginx.conf configuration file for errors
>> preventing nginx from successfully picking up your changes to
>> nginx.conf and (2) scrutinize nginx's access and error logs (you've
>> found these, right?) and Jetty's request logs
>> ($EXIST_HOME/tools/jetty/logs).  The idea is to ensure that nginx is
>> making the proxied request to the expected path for eXist.  (3) A
>> great troubleshooting tool is to log various request- and
>> context-related variables in in your controller.xql, using util:log or
>> console:log.  Good candidates for logging include: all of the $exist:*
>> variables, request:get-uri(), and
>> request:get-parameter('nginx-request-uri').  For example, see this use
>> of console:log in a controller.xql file:
>>
>> https://github.com/eXistSolutions/hsg-shell/blob/master/controller.xql#L26-L33.
>> Once you're sure that the requests are getting through to eXist at the
>> configured nginx & Jetty URLs, you may need to adjust aspects of your
>> eXist application that expect the `/exist/apps/myapp` URL context so
>> that links resolve correctly, whether you're hitting eXist directly or
>> via nginx.  For example, see
>>
>> https://github.com/eXistSolutions/hsg-shell/blob/master/modules/app.xqm#L9-L21.
>>
>> Joe
>
>
>
>
> --
> Pietro Maria Liuzzo
> cel (DE): +49 (0) 176 61 000 606
> Skype: pietro.liuzzo (Quingentole)
> https://uk.linkedin.com/in/pietroliuzzo

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open
Reply | Threaded
Open this post in threaded view
|

Re: exist as service and reverse proxying

Pietro Liuzzo
Dear Joe, 

thanks so much!

I have edited controller.xql to look like this

declare function local:get-uri() {
    (request:get-header("nginx-request-uri"), request:get-uri())[1]
};


if ($exist:path eq '') then
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
        <redirect url="{local:get-uri()}/"/>
    </dispatch>
    
else if ($exist:path eq "/") then
    (: forward root path to index.xql :)
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
        <redirect url="/index.html"/>
    </dispatch>

and it now works!

Thank you very much!
Pietro


On 12 July 2016 at 15:03, Joe Wicentowski <[hidden email]> wrote:
Hi Pietro,

Good, it sounds like you're getting close.  Check the redirects
leading to "index.html" in controller.xql.  My guess is that a leading
or trailing slash is getting added or removed in the redirect.

Joe

On Tue, Jul 12, 2016 at 6:53 AM, Pietro Liuzzo <[hidden email]> wrote:
> I have gone down to (2) for now and I think the path expected from Jetty is
> correct as it looks in the nginx.conf, but gets to Jetty from the log says
> doubled:
> exist/apps/myapp/
> arrives at Jetty as
> /exist/apps/myapp/exist/apps/myapp/index.html
>
> which clearly cannot be found.
>
> I think this happens within exist, but I am not sure where...
>
> Thanks for the further help!
>
>
>
>
> On 11 July 2016 at 20:03, Joe Wicentowski <[hidden email]> wrote:
>>
>> Hi Pietro,
>>
>> I'd expect your '/A/' example to work, so I would suggest (1) running
>> `nginx -t` to test your nginx.conf configuration file for errors
>> preventing nginx from successfully picking up your changes to
>> nginx.conf and (2) scrutinize nginx's access and error logs (you've
>> found these, right?) and Jetty's request logs
>> ($EXIST_HOME/tools/jetty/logs).  The idea is to ensure that nginx is
>> making the proxied request to the expected path for eXist.  (3) A
>> great troubleshooting tool is to log various request- and
>> context-related variables in in your controller.xql, using util:log or
>> console:log.  Good candidates for logging include: all of the $exist:*
>> variables, request:get-uri(), and
>> request:get-parameter('nginx-request-uri').  For example, see this use
>> of console:log in a controller.xql file:
>>
>> https://github.com/eXistSolutions/hsg-shell/blob/master/controller.xql#L26-L33.
>> Once you're sure that the requests are getting through to eXist at the
>> configured nginx & Jetty URLs, you may need to adjust aspects of your
>> eXist application that expect the `/exist/apps/myapp` URL context so
>> that links resolve correctly, whether you're hitting eXist directly or
>> via nginx.  For example, see
>>
>> https://github.com/eXistSolutions/hsg-shell/blob/master/modules/app.xqm#L9-L21.
>>
>> Joe
>
>
>
>
> --
> Pietro Maria Liuzzo
> cel (DE): <a href="tel:%2B49%20%280%29%20176%2061%20000%20606" value="+4917661000606">+49 (0) 176 61 000 606
> Skype: pietro.liuzzo (Quingentole)
> https://uk.linkedin.com/in/pietroliuzzo



--
Pietro Maria Liuzzo
cel (DE): +49 (0) 176 61 000 606
Skype: pietro.liuzzo (Quingentole)

------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
_______________________________________________
Exist-open mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/exist-open