Broken links in hg serve behind nginx via proxy_pass - mercurial

I have a simple Ubuntu 10.10, Nginx, Mercurial, PHP, and MySQL stack.
Current file structure:
/opt/nginx/html - document root for nginx (where my website files are)
/opt/nginx/html/.hg - repository data
/usr/share/mercurial/ - where the templates/static folder is
I am running hg -serve which is hosted at localhost:8000.
I do not have a clear understanding of how mercurial is linked to its resources after this point.
I have the following Nginx configuration for my desired entry point:
location /mercurial/ {
auth_basic "Authentication";
auth_basic_user_file /opt/nginx/html/.hg/file.password;
proxy_pass http://localhost:8000/;
}
This works fine except for fully loading the Mercurial Instance after authenticating. I get the page i expect to see but the /static references are all broken, and the links I click on such as one of the changesets is broken as well (they are not found or take me back to the broken main page). It tries to find these files at my-ip-address/static/... which is incorrect.
The mercurial ui directory is outside of my document root /opt/nginx/html.
The path of where /static actually is: /usr/share/mercurial/templates
Is this the problem?? That the files are outside of my nginx document root?
When mercurial runs where is /rev and how can i make sure that mercurial runs correctly with the correct resource URLs?
Note: Just going to my-ip-address:8000 works 100%. Nothing is broken and no css or js is missing. The problem is happening with regards to the proxy_pass.
How can I have these files properly linked up? Is a rewrite required?

The problem is that by default, the web server launched with hg serve doesn't add any prefix to the URLs it generates. It has no way to know that you're NGinx proxy pass request for the /mercurial location.
You must set the prefix to use with the --prefix option. hg help serve tells us this about the option :
--prefix PREFIX prefix path to serve from (default: server root)
So just run the server this way : hg serve --prefix mercurial and everything should be fine ! (don't forget to remove your alias...)
If you prefer, the prefix option can also be set in the configuration file if you have one :
[web]
prefix = mercurial

Soultion: I needed to add the following to my NGINX config file;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Related

Flask and Nginx: css not recognized

I am trying to deploy my website using Flask. Everything is working perfectly locally (localhost:5000), but when I deploy it on my distant linux ubuntu server (www.linode.com) I encouter a problem. I have installed gunicorn and nginx, but my CSS files are not taken into account, thus, my webpage just appears as an HTML content alone, without shape/colors and so on.
I am very new at web development, and I don't understand why it works perfectly locally and not anymore (not totally) on the linux server. Is the problem related to the nginx that can't manage to do the link between HTML and CSS? Because locally I don't need to use nginx, and everything works.
Here is my nginx config file for when I type sudo nano /etc/nginx/sites-enabled/assets:
server {
listen 80;
server_name 45.79.250.111;
include /etc/nginx/mime.types;
location /static {
alias /home/gardy/ladybird_site/assets/static;
}
location / {
proxy_pass http://localhost:8000;
include /etc/nginx/proxy_params;
proxy_redirect off;
}
}
And here is my website directory tree:
ladybird_site/
requirements.txt
run.py
assets/
__init__.py
config.py
models.py
fonts/
et-line.eot
...
fontawesome-webfont.woff2
main/
routes.py
__init__.py
templates/
index.html
layout.html
static/
css/
animate.css
bootstrap.min.css
font-awesome.min.css
style.css
bxslider/
jquery.bxslider.css
images/
bx_loader.gif
controls.png
et-line-font/
style.css
magnific-popup/
magnific-popup.css
owl-carousel/
owl.carousel.css
owl.theme.css
owl.transitions.css
images/
ldb_ico.ico
founder_pics/
custom_pics/
f1.jpg
f2.jpg
f3.jpg
ldb_imgs/
1.png
2.png
3.png
js/
bootstrap-hover-dropdown.min.js
bootstrap.min.js
...
jquery.bxslider.min.js
main.js
I am getting crazy with that stuff, trying and modifying things since two days without significant improvments, and found no answers on the internet...
Thanks a lot
Your static path looks fine. You have set your server to listen to port 80, which is the default port for HTTP, but maybe you have not opened it. The next step is for you to open it.
sudo ufw allow http/tcp
You mention that your server is still listening to port 5000. However, if you are done with testing, you can disallow this port from being used by Nginx.
sudo ufw delete allow 5000
Enable these new rules by running the command below:
sudo ufw enable
# Hit 'y' for 'yes' when prompted
Finally, you can now restart your Nginx server:
sudo systemctl status nginx.service
If you navigate to your application's IP address, and append static/css/style.css, you should be able to access your CSS file. I mean http://<your IP address>/static/css/style.css

Virtualhost url not found

I installed LAMP on a new EC2 server and via the public IP address I can see the default Ubuntu webpage as loaded by my new apache server.
My problems began when I created a new directory, simply called test, alongside the original html directory. In test, I created index.html with the content "Test page responding and loading ...".
So, I then copy /etc/apache2/sites-available/000-default.conf to test.conf and set DocumentRoot /var/www/test.
I enabled the virtualhost by sudo a2ensite test.conf and restarted the server with sudo service apache2 reload.
In my browser, when I visit the IP address I get the ubuntu page as loaded by apache. I then append /test to the end of the IP address and I get a Not found: The requested URL /test was not found on this server. error.
I disabled the 000-default.conf virtualhost, just out of curiosity, and restarted the server. My output from test/index.html loads in a browser using just the IP address - which I did not expect.
ServerAdmin webmaster#localhost
DocumentRoot /var/www/test
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
I'm trying to simply serve test/index.html when I visit .../test in my browser.
If you don't want Apache to select the default virtual host (the first one for a given host+port) you'll need to use ServerName or ServerAlias in your added virtual host to make sure it matches the hostname/address you're using in the browser.
apachectl -S will summarize your vhosts.

Have index.html file but still getting a directory listing

I have an index.html file in my Apache DocumentRoot directory but when I go to my URL, I am still getting a directory listing of my DocumentRoot directory instead of the index.html file being displayed. The apache access_log shows 200's when I reload the page. Any suggestions?
Use
DirectoryIndex index.html
It tells apache what document to show for a directory request.
update
You should specify just the filename that apache will look for in the folder requested.
Not saying this will fix it for you, but for me when first getting started with Apache2 it was file permissions that would get forgotten when moving or writing new file under the web root directory
ls -hal /var/www/host_one/index.html
If above doesn't have read (r) permissions for the same user:group or if the ownership doesn't include the user/group of the web server, then try the following for allowing group reads
# Modify ownership, change 'www_host' to Apache2 group
chown ${USER}:www_host /var/www/host_one/index.html
# give read+write (6) to user and read (4) to group owners
chmod 640 /var/www/host_one/index.html
Try refreshing the website and see if permissions where the issue. Note most web documents only require read permissions and ownership to be correct for browsers to be allowed to pick them up for rendering, on rare occasions you may need executable (1 or x) permissions for server scripts (be cautious of ownership in such cases) and last write permissions (2 or w) should likely never be seen without good reasons on files within your web root.
Second thing to try, use the index.html within your browsers URL bar
# by IP
http://192.168.0.100/index.html
# by domain
http://site-name.local/index.html
If the above loaded your document then, like #Pekka 웃 stated already, you've likely got a server option that's missing or enabling directory listings instead of looking for a index page within that directory. If this is the problem then there's two ways of fixing it that I've tried in the past. One, htaccess configuration to disable directory listing within that sub-directory, two, server vhost configuration to prevent whole site from directory listings. Personally I prefer to use option two and then on directories that should be allowed to be listed place an htaccess config for permissions instead of denials.

How to serve any static directory containing web pages in django?

I'm working on an application where we want the following to happen:
The administrators upload zip archives containing web content (whole static websites) through admin interface
The archive is unzipped in the background to a random directory
The static content is served at a url that looks like http://<host>/site/<user-friendly-url>/, with the index.html at the root, all the paths to css and js files, etc.
I know how to configure the route, but I was wondering what to put in the view. I have tried with STATIC_{ROOT,URL} and MEDIA_{ROOT,URL} but I feel it's not the right way.
In summary : is there a way to put something like return serve_this_static_directory() in the view? or is there any workaround there?
Thanks for your answers
edit:
Trying to clarify my question.
The url (http://<host>/site/<user-friendly-url>/) doesn't match the directory name (which could be anything, like XYZ_html/).
Administrators must be able to upload new archives, which should be extracted and served automatically.
For those two reasons, I don't have any idea how to serve this content using the webserver, because how is it possible to configure the routes then?
First you must realize that django won't serve your static or media files by default, you should use your HTTP server (nginx, apache or other) for that.
Next thing to do is to configure your HTTP server to serve "/path/to/your/project/public" and any files inside on root URL of your domain, and instead of serving 404 error if file is not found, it should redirect to django (so if there is some file in path /site/something/ your HTTP server should serve that file, if not, it shoudl serve anything that django will output on that path. Sample configuration for nginx might look like this:
server {
listen 80;
server_name oskar.local;
root /path/to/your/project/public;
location #default {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
include /etc/nginx/uwsgi_params;
uwsgi_pass app_server; # or some proxy_pass if you're not using uwsgi
break;
}
location / {
try_files $uri #default;
}
}
And last step is to configure your django app's MEDIA_* settings:
MEDIA_ROOT = "/path/to/your/project/public"
MEDIA_URL = "/"
But be careful! With that approach there is potential vulnerability: any user can put files into path that is normally handled by django. Better approach will be to put all of "sub-sites" inside some sub-directory or into separate domain.

Wildcard Subdomains

I know there have been a few threads on this before, but I have tried absolutely everything suggested (that I could find) and nothing has worked for me thus far...
With that in mind, here is what I'm trying to do:
First, I want to allow users to publish pages and give them each a subdomain of their choice (ex: user.example.com). From what I can gather, the best way to do this is to map user.example.com to example.com/user with mod_rewrite and .htaccess - is that correct?
If that is correct, can somebody give me explicit instructions on how to do this?
Also, I am doing all of my development locally, using MAMP, so if somebody could tell me how to set up my local environment to work in the same manner (I've read this is more difficult), I would greatly appreciate it. Honestly, I have been trying a everything to no avail, and since this is my first time doing something like this, I am completely lost.
Some of these answers have been REALLY helpful, but for the system I have in mind, manually adding a subdomain for each user is not an option. What I'm really asking is how to do this on the fly, and redirect wildcard.example.com to example.com/wildcard -- the way Tumblr is set up is a perfect example of what I'd like to do.
As far as how to set up the DNS subdomain wildcard, that would be a function of your DNS hosting provider. This would be different steps depending on which hosting provider you have and would be a better question for them.
Once you've set that up with the DNS host, from your web app you really are just URL rewriting, which can be done with some sort of module for the web server itself, such as isapi rewrite if you're on IIS (this would be the preferred route if possible). You could also handle rewriting at the application level as well (like using routing if on ASP.NET).
You'd rewrite the URL so http://myname.example.com would become http://example.com/something.aspx?name=myname or something. From there on out, you just handle it as if the myname value was in the query string as normal. Does that make sense? Hope I didn't misunderstand what you're after.
I am not suggesting that you create a subdomain for each user, but instead create a wildcard subdomain for the domain itself, so anything.example.com (basically *.example.com) goes to your site. I have several domains setup with MyDomain. Their instructions for setting this up is like this:
Yes, you can configure a wild card but
it will only work if you set it up as
an A Record. Wildcards do not work
with a C Name. To use a wildcard, you
use the asterisks character ''. For
example, if you create and A Record
using a wild card, *.example.com,
anything that is entered in the place
where the '' is located, will resolve
to the specified IP address. So if you
enter 'www', 'ftp', 'site', or
anything else before the domain name,
it will always resolve to the IP
address
I have some that are setup in just this way, having *.example.com go to my site. I then can read the base URL in my web app to see that ryan.example.com is what was currently accessed, or that bill.example.com is what was used. I can then either:
Use URL rewriting so that the subdomain becomes a part of the query string OR
Simply read the host value from the accessed URL and perform some logic based on that value.
Does that make sense? I have several sites set up in just this exact way: create the wildcard for the domain with the DNS host and then simply read the host, or base domain from the URL to decide what to display based on the subdomain (which was actually a username)
Edit 2:
There is no way to do this without a DNS entry. The "online world" needs to know that name1.example.com, name2.example.com,..., nameN.example.com all go to the IP address for your server. The only way to do this is with the appropriate DNS entry. You have to add the wildcard DNS entry for your domain with your DNS host. Then it's just a matter of you reading the subdomain from the URL and taking the appropriate action in your code.
The best thing to do if you are running *AMP is to do what Thomas suggests and do virtual hosts in Apache. You can do this either with or without the redirect you describe.
Virtual hosts
Most likely you will want to do name-based virtual hosts, as it's easiest to set up and only requires one IP address (so will also be easy to set up and test on your local MAMP machine). IP-based virtual hosts is better in some other respects, but you have to have an IP address for each domain.
This Wikipedia page discusses the differences and links to a good basic walk-thru of how to do name-based vhosts at the bottom.
On your local machine for testing, you'll also have to set up fake DNS names in /etc/hosts for your fake test domain names. i.e. if you have Apache listening on localhost and set up vhost1.test.domain and vhost2.test.domain in your Apache configs, you'd just add these domains to the 127.0.0.1 line in /etc/hosts, after localhost:
127.0.0.1 localhost vhost1.test.domain vhost2.test.domain
Once you've done the /etc/hosts edit and added the name-based virtual host configs to your Apache configuration file(s), that's it, restart Apache and your test domains should work.
Redirect with mod_rewrite
If you want to do redirects with mod_rewrite (so that user.example.com isn't directly hosted and instead redirects to example.com/user), then you will also need to do a RewriteCond to match the subdomain and redirect it:
RewriteEngine On
RewriteCond %{HTTP_HOST} ^subdomain\.example\.com
RewriteRule ^(.*)$ http://example.com/subdomain$1 [R]
You can put this in a .htaccess or in your main Apache config.
You will need to add a pair of rules like the last two for each subdomain you want to redirect. Or, you may be able to capture the subdomain in a RewriteCond to be able to use one wildcard rule to redirect *.example.com to example.com/
-- but that smells really bad to me from a security standpoint.
All together, vhosts and redirect
It's better to be more explicit and set up a virtual host configuration section for each hostname you want to listen for, and put the rewrite rules for each of these hostnames inside its virtual host config. (It is always more secure and faster to put this kind of stuff inside your Apache config and not .htaccess, if you can help it -- .htaccess slows performance because Apache is constantly scouring the filesystem for .htaccess files and reparsing them, and it's less secure because these can be screwed up by users.)
All together like that, the vhost config inside your Apache configs would be:
NameVirtualHost 127.0.0.1:80
# Your "default" configuration must go first
<VirtualHost 127.0.0.1:80>
ServerName example.com
ServerAlias www.example.com
DocumentRoot /www/siteroot
# etc.
</VirtualHost>
# First subdomain you want to redirect
<VirtualHost 127.0.0.1:80>
ServerName vhost1.example.com
RewriteEngine On
RewriteRule ^(.*)$ http://example.com/vhost1$1 [R]
</VirtualHost>
# Second subdomain you want to redirect
<VirtualHost 127.0.0.1:80>
ServerName vhost2.example.com
RewriteEngine On
RewriteRule ^(.*)$ http://example.com/vhost2$1 [R]
</VirtualHost>
I realize that I'm pretty late responding to this question, but I had the same problem in regards to a local development solution. In another SO thread I found better solutions and thought I would share them for anyone with the same question in the future:
VMware owned wild card domain that resolves any subdomain to 127.0.0.1:
vcap.me resolves to 127.0.0.1
www.vcap.me resolves to 127.0.0.1
or for more versatility 37 Signals owns a domain to map any subdomain to any given IP using a specific format:
127.0.0.1.xip.io resolves to 127.0.0.1
www.127.0.0.1.xip.io resolves to 127.0.0.1
db.192.168.0.1.xip.io resolves to 192.168.0.1
see xip.io for more info
I am on Ubuntu 16.04 and since 14.04 I've using solution provided by Dave Evans here and it works fine for me.
Install dnsmasq
sudo apt-get install dnsmasq
Create new file localhost.conf under /etc/dnsmasq.d dir with the following line
#file /etc/dnsmasq.d/localhost.conf
address=/localhost/127.0.0.1
Edit /etc/dhcp/dhclient.conf and add the following line
prepend domain-name-servers 127.0.0.1;
(You’ll probably find that this line is already there and you just need to uncomment it.)
Last one is restart the service
sudo systemctl restart dnsmasq
sudo dhclient
Finally, you should check if it's working.
dig whatever.localhost
note:
If you want to use it on your web server, you need to simply change the 127.0.0.0 to your actual IP address.
I had to do exactly the same for one of my sites. You can follow the following steps
If you've cPanel on your server, create a subdomain *, if not, you'd have to set-up an A record in your DNS (for BIND see http://ma.tt/2003/10/wildcard-dns-and-sub-domains/). On your dev. server you'd be far better off faking subdomains by adding each to your hosts file.
(If you used cPanel you won't have to do this). You'll have to add soemthing like the following to your apache vhosts file. It largely depends on what type of server (shared or not) you're running. THE FOLLOWING CODE IS NOT COMPLETE. IT'S JUST TO GIVE DIRECTION. NOTE: ServerAlias example.com *.example.com is important.
<VirtualHost 127.0.0.1:80>
DocumentRoot /var/www/
ServerName example.com
ServerAlias example.com *.example.com
</VirtualHost>
Next, since you can use the PHP script to check the "Host" header and find out the subdomain and serve content accordingly.
First, I want to allow users to
publish pages and give them each a
subdomain of their choice (ex:
user.mysite.com). From what I can
gather, the best way to do this is to
map user.mysite.com to mysite.com/user
with mod_rewrite and .htaccess - is
that correct?
You may be better off using virtual hosts. That way, each user can have a webserver configuration pretty much independent of others.
The syntax goes something like this:
<VirtualHost *:80>
DocumentRoot /var/www/user
ServerName user.mysite.com
...
</VirtualHost>
From what I have seen on many webhosts, they setup a virtual host on apache.
So if your www.mysite.com is served from /var/www, you could create a folder for each user. Then map the virtual host to that folder.
With that, both mysite.com/user and user.mysite.com works.
As for your test enviroment, if you are on windows, I would suggest editing your HOSTS file to map mysite.com to your local PC (127.0.0.1), as well as any subdomains you set up for testing.
The solution I found for Ubuntu 18.04 is similar to this one but involves NetworkManager config:
Edit the file /etc/NetworkManager/NetworkManager.conf, and add the line dns=dnsmasq to the [main] section
sudo editor /etc/NetworkManager/NetworkManager.conf
should look like this:
[main]
plugins=ifupdown,keyfile
dns=dnsmasq
...
Start using NetworkManager's resolv.conf
sudo rm /etc/resolv.conf
sudo ln -s /var/run/NetworkManager/resolv.conf /etc/resolv.conf
Create a file with your wildcard configuration
echo 'address=/.localhost/127.0.0.1' | sudo tee /etc/NetworkManager/dnsmasq.d/localhost-wildcard.conf
Reload NetworkManager configuration
sudo systemctl reload NetworkManager
Test it
dig localdomain.localhost
You can also add any other domain, quite useful for some types of authentication when using a local development setup.
echo 'address=/.local-dev.workdomain.com/127.0.0.1' | sudo tee /etc/NetworkManager/dnsmasq.d/workdomain-wildcard.conf
Then this works:
dig petproject.local-dev.workdomain.com
;; ANSWER SECTION:
petproject.local-dev.workdomain.com. 0 IN A 127.0.0.1