Member Avatar for PrometheanSin

after pulling my hair out with this one, I'm afraid I'm going to have to turn the knowledge of those much wiser.....again!

ok so here's the problem, I am following the following two railscasts:

deploying to a vps
asset pipeline in production

My setup:
Ubuntu Server 12.04
nginx
unicorn
ruby 1.9.3p125
rails 3.2.2

Ad far as I can tell everything has been followed to a T (or Tea or maybe a Tee, I've never seen this phrase written down before). The site is now running on my server but the stylesheet and javascript files are not loaded. it seems to be the same problem as Rails 3.1 404 Error when loading .css files that are present in public/assets or Rails 404 error for Stylesheet or JavaScript files. While the first has no answer, the latter suggests it may be a problem with the web-server not looking for static resources and perhaps shouldn't.

Does anybody know what might be causing this?

I'll try to add the relative files, but I have run out of room in the comment box

  • 1 Contributor
  • 5 Replies
  • 768 Views
  • 40 Minutes Discussion Span
  • Latest Post Latest Post by PrometheanSin

Dani AI

Generated

Brief diagnosis and practical next steps (builds on 's followup)

This is the classic post-deploy asset 404: the fingerprinted files exist on disk but requests either never hit Nginx's static-handler or they get proxied to Unicorn/Rails which then returns 404 or compiles assets on demand. In this thread solved it by fixing Unicorn restarts — that is an important clue. When a deploy updates the current symlink but the app server is not restarted (or not upgraded correctly), the running Unicorn workers can still be using an old release, manifest, or middleware stack so static requests behave differently.

Quick checklist to reproduce and fix the problem

  • Confirm Nginx is actually trying to serve /assets/... before proxying to Unicorn (location ordering / try_files).
  • Look for Sprockets runtime compilation in the Rails log; if you see many "Compiled ..." messages, asset requests are reaching Rails. That indicates Nginx is not serving the files.
  • Check Nginx error log for "open() ... failed" lines to catch wrong root, missing module, or permission issues.
  • Verify file permissions so the Nginx worker user can read the files.
  • Ensure Capistrano (or your deploy hook) does a proper Unicorn restart/upgrade so workers use the new release and new manifest.

Useful commands to run on the server

# check HTTP response for a fingerprinted asset
curl -I http://your.host/assets/application-<digest>.css

# inspect nginx errors
sudo tail -n 200 /var/log/nginx/error.log

# find and fix ownership/permissions for assets (adjust user/app)
sudo chown -R www-data:www-data /home/<user>/apps/<app>/current/public/assets
sudo find /home/<user>/apps/<app>/current/public/assets -type d -exec chmod 755 {} \;
sudo find /home/<user>/apps/<app>/current/public/assets -type f -exec chmod 644 {} \;

# check unicorn processes and pid
ps aux | grep '[u]nicorn'

Final notes
Do not rely on config.assets.compile = true in production — it hides the root cause and kills performance. Make sure your deploy script actually restarts or performs a zero-downtime upgrade of Unicorn (test the init script manually), confirm Nginx's static assets location is effective, and then re-deploy to verify static files are served by Nginx.

Member Avatar for PrometheanSin

here is a dump of the production .log on my server

some facts:

  • Firstly, I was getting some nasty errors about pre-compilation and noticed that my individual css/js files had been minified into their application counterparts and so I changed their declarations in the ruby helpers to reflect this(<%= stylesheet_link_tag "application" %>).

  • The application works locally when I change to config.serve_static_assets = true in config/environments/production.rb

    Started GET "/main" for 92.237.28.149 at 2012-07-09 23:08:02 +0100
    Processing by MainController#index as HTML
    Rendered shared/_login_form.html.erb (0.4ms)
    Rendered shared/_footer.html.erb (0.4ms)
    Rendered shared/_details_form.html.erb (7.7ms)
    Rendered main/index.html.erb within layouts/main_logged (17.1ms)
    Completed 200 OK in 21ms (Views: 20.5ms | ActiveRecord: 0.0ms)
    Compiled jquery.js (2ms) (pid 8253)
    Compiled jquery_ujs.js (0ms) (pid 8253)
    Compiled admin_user.js (227ms) (pid 8253)
    Compiled construction.js (227ms) (pid 8253)
    Compiled google-map.js (0ms) (pid 8253)
    Compiled jquery-1.2.6.min.js (1ms) (pid 8253)
    Compiled jquery-ui-personalized-1.5.2.packed.js (0ms) (pid 8253)
    Compiled login.js (227ms) (pid 8253)
    Compiled main.js (222ms) (pid 8253)
    Compiled menu.js (223ms) (pid 8253)
    Compiled order_steps.js (225ms) (pid 8253)
    Compiled register_user.js (223ms) (pid 8253)
    Compiled sprinkle.js (0ms) (pid 8253)
    Compiled sticky_basket/jquery.easing.1.3.js (0ms) (pid 8253)
    Compiled sticky_basket/stickysidebar.jquery.js (0ms) (pid 8253)
    Compiled sticky_basket/stickysidebar.jquery.min.js (0ms) (pid 8253)
    Compiled user.js (292ms) (pid 8253)
    Compiled application.js (2155ms) (pid 8253)
    Compiled admin_user.css (12ms) (pid 8253)
    Compiled construction.css (0ms) (pid 8253)
    Compiled construction.css (1ms) (pid 8253)
    Compiled lightbox.css (0ms) (pid 8253)
    Compiled login.css (1ms) (pid 8253)
    Compiled main.css (0ms) (pid 8253)
    Compiled login.css (1ms) (pid 8253)
    Compiled main.css (0ms) (pid 8253)
    Compiled main.css (2ms) (pid 8253)
    Compiled menu.css (1ms) (pid 8253)
    Compiled menu_style.css (0ms) (pid 8253)
    Compiled normalise.css (0ms) (pid 8253)
    Compiled order_steps.css (0ms) (pid 8253)
    Compiled order_steps.css (1ms) (pid 8253)
    Compiled register_user copy.css (0ms) (pid 8253)
    Compiled register_user.css (0ms) (pid 8253)
    Compiled register_user.css (1ms) (pid 8253)
    Compiled tab_style.css (0ms) (pid 8253)
    Compiled user.css (1ms) (pid 8253)
    Compiled application.css (236ms) (pid 8253)
    Compiled jquery.js (2ms) (pid 8253)
    Compiled jquery_ujs.js (0ms) (pid 8253)
    Compiled admin_user.js (273ms) (pid 8253)
    Compiled construction.js (225ms) (pid 8253)
    Compiled google-map.js (0ms) (pid 8253)
    Compiled jquery-1.2.6.min.js (1ms) (pid 8253)
    Compiled jquery-ui-personalized-1.5.2.packed.js (0ms) (pid 8253)
    Compiled login.js (269ms) (pid 8253)
    Compiled main.js (225ms) (pid 8253)
    Compiled menu.js (224ms) (pid 8253)
    Compiled order_steps.js (226ms) (pid 8253)
    Compiled register_user.js (224ms) (pid 8253)
    Compiled sprinkle.js (0ms) (pid 8253)
    Compiled sticky_basket/jquery.easing.1.3.js (0ms) (pid 8253)
    Compiled sticky_basket/stickysidebar.jquery.js (0ms) (pid 8253)
    Compiled sticky_basket/stickysidebar.jquery.min.js (0ms) (pid 8253)
    Compiled user.js (224ms) (pid 8253)
    Compiled application.js (2096ms) (pid 8253)
    Compiled admin_user.css (12ms) (pid 8253)
    Compiled construction.css (0ms) (pid 8253)
    Compiled construction.css (1ms) (pid 8253)
    Compiled lightbox.css (0ms) (pid 8253)
    Compiled login.css (1ms) (pid 8253)
    Compiled main.css (0ms) (pid 8253)
    Compiled login.css (1ms) (pid 8253)
    Compiled main.css (0ms) (pid 8253)
    Compiled main.css (1ms) (pid 8253)
    Compiled menu.css (2ms) (pid 8253)
    Compiled menu_style.css (0ms) (pid 8253)
    Compiled normalise.css (0ms) (pid 8253)
    Compiled order_steps.css (0ms) (pid 8253)
    Compiled order_steps.css (1ms) (pid 8253)
    Compiled register_user copy.css (0ms) (pid 8253)
    Compiled register_user.css (0ms) (pid 8253)
    Compiled register_user.css (1ms) (pid 8253)
    Compiled tab_style.css (0ms) (pid 8253)
    Compiled user.css (1ms) (pid 8253)
    Compiled application.css (179ms) (pid 8253)

    Started GET "/main" for 92.237.28.149 at 2012-07-09 23:15:00 +0100
    Processing by MainController#index as HTML
    Rendered shared/_login_form.html.erb (1.2ms)
    Rendered shared/_footer.html.erb (1.1ms)
    Rendered shared/_details_form.html.erb (15.5ms)
    Rendered main/index.html.erb within layouts/main_logged (80.9ms)
    Completed 200 OK in 142ms (Views: 140.4ms | ActiveRecord: 0.0ms)

    Started GET "/main" for 92.237.28.149 at 2012-07-09 23:33:54 +0100
    Processing by MainController#index as HTML
    Rendered shared/_login_form.html.erb (1.2ms)
    Rendered shared/_footer.html.erb (1.1ms)
    Rendered shared/_details_form.html.erb (15.6ms)
    Rendered main/index.html.erb within layouts/main_logged (80.7ms)
    Completed 200 OK in 142ms (Views: 140.0ms | ActiveRecord: 0.0ms)

As you can see there are no more complaints about pre-compilation

Member Avatar for PrometheanSin

here is the html generated

<title>HTML Snippet</title>
  <link href="/assets/application-dd445a707b7e96c914afd14bf1f02bbc.css" media="screen" rel="stylesheet" type="text/css" />
  <script src="/assets/application-def3878140cda305cb0ca5ef745fd758.js" type="text/javascript"></script>
  <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5.1/jquery.min.js" type="text/javascript"></script>
  <script src="" type="text/javascript"></script>

These links generate an nginx 404 if I click on them but if i change the url to http://<ipaddress>/public/assets/application-dd445a707b7e96c914afd14bf1f02bbc.css i get a rails generated The page you were looking for doesn't exist.

And here is the important (I think) contents of /public/assets on my server

application-7c90c01effcf9d8593e28e4c4b716ac2.css
application-7c90c01effcf9d8593e28e4c4b716ac2.css.gz
application.css
application.css.gz
application-dd445a707b7e96c914afd14bf1f02bbc.css
application-dd445a707b7e96c914afd14bf1f02bbc.css.gz
application-def3878140cda305cb0ca5ef745fd758.js
application-def3878140cda305cb0ca5ef745fd758.js.gz
application.js
application.js.gz
Member Avatar for PrometheanSin

finally, the area that I'm not so clear on and just so its all covered, here are the files used in the railscast for configuration of capistrano, nginx and unicorn:

I have uncommented the load 'deploy/assets' in capfile

deploy.rb

require "bundler/capistrano"

server "<myserverip>", :web, :app, :db, primary: true

set :application, "little_oven_deploy"
set :user, "daz"
set :deploy_to, "/home/#{user}/apps/#{application}"
set :deploy_via, :remote_cache
set :use_sudo, false

set :scm, "git"
set :repository, "<mygitrepo>"
set :branch, "master"

default_run_options[:pty] = true
ssh_options[:forward_agent] = true

after "deploy", "deploy:cleanup" # keep only the last 5 releases

namespace :deploy do
  %w[start stop restart].each do |command|
    desc "#{command} unicorn server"
    task command, roles: :app, except: {no_release: true} do
      run "/etc/init.d/unicorn_#{application} #{command}"
    end
  end

  task :setup_config, roles: :app do
    sudo "ln -nfs #{current_path}/config/nginx.conf /etc/nginx/sites-enabled/#{application}"
    sudo "ln -nfs #{current_path}/config/unicorn_init.sh /etc/init.d/unicorn_#{application}"
    run "mkdir -p #{shared_path}/config"
    put File.read("config/database.example.yml"), "#{shared_path}/config/database.yml"
    puts "Now edit the config files in #{shared_path}."
  end
  after "deploy:setup", "deploy:setup_config"

  task :symlink_config, roles: :app do
    run "ln -nfs #{shared_path}/config/database.yml #{release_path}/config/database.yml"
  end
  after "deploy:finalize_update", "deploy:symlink_config"

  desc "Make sure local git is in sync with remote."
  task :check_revision, roles: :web do
    unless `git rev-parse HEAD` == `git rev-parse origin/master`
      puts "WARNING: HEAD is not the same as origin/master"
      puts "Run `git push` to sync changes."
      exit
    end
  end
  before "deploy", "deploy:check_revision"
end

nginx.conf

upstream unicorn {
  server unix:/tmp/unicorn.little_oven_deploy.sock fail_timeout=0;
}

server {
  listen 80 default deferred;
  # server_name example.com;
  root /home/daz/apps/little_oven_deploy/current/public

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @unicorn;
  location @unicorn {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass 
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 4G;
  keepalive_timeout 10;
}
unicorn.rb

root = "/home/daz/apps/little_oven_deploy/current"
working_directory root
pid "#{root}/tmp/pids/unicorn.pid"
stderr_path "#{root}/log/unicorn.log"
stdout_path "#{root}/log/unicorn.log"

listen "/tmp/unicorn.little_oven_deploy.sock"
worker_processes 2
timeout 30
Edited by PrometheanSin because: changes to outdated line
Member Avatar for PrometheanSin

unicorn_init.sh

#!/bin/sh
### BEGIN INIT INFO
# Provides:          unicorn
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Manage unicorn server
# Description:       Start, stop, restart unicorn server for a specific application.
### END INIT INFO
set -e

# Feel free to change any of the following variables for your app:
TIMEOUT=${TIMEOUT-60}
APP_ROOT=/home/daz/apps/little_oven_deploy/current
PID=$APP_ROOT/tmp/pids/unicorn.pid
CMD="cd $APP_ROOT; bundle exec unicorn -D -c $APP_ROOT/config/unicorn.rb -E production"
AS_USER=daz
set -u

OLD_PIN="$PID.oldbin"

sig () {
  test -s "$PID" && kill -$1 `cat $PID`
}

oldsig () {
  test -s $OLD_PIN && kill -$1 `cat $OLD_PIN`
}

run () {
  if [ "$(id -un)" = "$AS_USER" ]; then
    eval $1
  else
    su -c "$1" - $AS_USER
  fi
}

case "$1" in
start)
  sig 0 && echo >&2 "Already running" && exit 0
  run "$CMD"
  ;;
stop)
  sig QUIT && exit 0
  echo >&2 "Not running"
  ;;
force-stop)
  sig TERM && exit 0
  echo >&2 "Not running"
  ;;
restart|reload)
  sig HUP && echo reloaded OK && exit 0
  echo >&2 "Couldn't reload, starting '$CMD' instead"
  run "$CMD"
  ;;
upgrade)
  if sig USR2 && sleep 2 && sig 0 && oldsig QUIT
  then
    n=$TIMEOUT
    while test -s $OLD_PIN && test $n -ge 0
    do
      printf '.' && sleep 1 && n=$(( $n - 1 ))
    done
    echo

    if test $n -lt 0 && test -s $OLD_PIN
    then
      echo >&2 "$OLD_PIN still exists after $TIMEOUT seconds"
      exit 1
    fi
    exit 0
  fi
  echo >&2 "Couldn't upgrade, starting '$CMD' instead"
  run "$CMD"
  ;;
reopen-logs)
  sig USR1
  ;;
*)
  echo >&2 "Usage: $0 <start|stop|restart|upgrade|force-stop|reopen-logs>"
  exit 1
  ;;
esac

unicorn_init.sh is also chmod +x.

Sorry for the potential overload of information guys but if it helps give you a snapshot of my issue, it helps to get a diagnosis and any help is much appreciated.

Thanks in advance

Member Avatar for PrometheanSin

not quite sure how to do this (I'm not sure on the etiquette of answering your own question) but I may have resolved the prpoblem, basically, I figured that although the nginx server was restarting, the unicorn server was not (or if so the effects were not taking effect). I found the solution to this post Restart Unicorn issue (capistrano) seems to have done the trick. now i just need to figure out how to wire it into my code!

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.