Courant's compute / assignment servers are shared, so other students will be running their projects on it. Consequently, to run your node web application, you'll have run it on a port number and server that's been assigned to you.

There are multiple potential servers that can host assignments. See the documentation on compute servers. These can only be accessed via ssh by going through another host, cims.nyu.edu.

To determine which port number and server you'll use:

1. Log in to NYU Classes
2. Go to Assignments
3. Click on Milestone 2
4. Find the link to the port retrieval page in the milestone 2 instructions

We'll be using ssh, a commandline tool that allows you to login to and run commands on a remote server, to access your server. If you're on windows, a popular ssh client is putty. You'll need a "cims" account to log in to any of these compute/assignment servers. You should have one already created for you.

1. You can retrieve your credentials by going to https://cims.nyu.edu/webapps/password first
   • use your netid to log in
   • save the credentials that you receive
   • if the above does not work or if you need to reset the password that was given to you from the above page, you can go to https://cims.nyu.edu/webapps/password/reset * if you receive an error saying that your account does not exist:
   • try using this flowchart to troubleshoot: https://cims.nyu.edu/cms_content/cims-account-access-troubleshooting.pdf
     • yes, I know it looks complicated, but follow along to make sure there's really a system related issue
   • if that still doesn't work, send me a note with:
     • all of the solutions you've tried
     • along with your username and netid (do not send me your password)
   • I will contact helpdesk@ on your behalf
2. To access your assigned server, you must first ssh to access.cims.nyu.edu… and then from there, ssh to your actual server:
   • ssh your_username@access.cims.nyu.edu
   • once you've logged in… use ssh again
   • ssh your_username@your_assigned_server
3. Once you've logged in, create the following directories in your home directory:
   • var/log a directory to dump your application logs
   • usr/local/lib to store all of your "global" node modules
   • opt … as the directory where you'll install your web application
   • run the following command to create the directories specified above (note that ~ means your home directory):
     

   mkdir -p ~/var/log
   mkdir -p ~/usr/local/lib
   mkdir ~/opt

4. Verify that these directories exist by running:

ls ~

Should show:

opt  usr  var

1. The version of node on the servers currently doesn't support ES6.
   • consequently, you'll have to install the most current version of node
   • you can do this by installing and using a tool called node version manager, or nvm
   • to install:
     • curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.1/install.sh | bash
   • to get the latest version of node
     • . ~/.nvm/nvm.sh
     • nvm install node

You'll have your own database on an instance of mongod running on one of Courant's server. You don't have to keep the database running like you do locally; that's already taken care of for you.

1. Follow these instructions for initializing your mongoDB database on the server
2. Make sure you keep the password that's given to you after you follow the above instructions. (ideally, in a password safe)
3. To test connectivity, you can try connecting to your database while your ssh'd in to the server:

module load mongodb-3.2.0
mongo <dbname> --host class-mongodb.cims.nyu.edu -u <username> -p

1. You should see your MongoDB shell if you're able to successfully connect!
2. If you see the following error Failed global initialization: BadValue Invalid or no user locale set. Please ensure LANG and/or LC_* environment variables are set correctly. … try setting your locale (default language - like English) with export LC_ALL=C before running the mongo commandline client on the server

Because the mongodb server you'll connect to from the server requires user authentication, the string you pass in to mongoose.connect (your database connection string) will have to include the username and password you were given from the previous section, Part 3. The new connection string will have the following format:

mongodb://USERNAME:PASSWORD@class-mongodb.cims.nyu.edu/USERNAME

Where:

• USERNAME - is the username you used for logging it to the server
• PASSWORD - is the password for mongodb that you created from Part 3.

The name of your database is the same as your username!

However, you should not put these credentials directly into your db.js file, and they should not be in a file in version control (you may inadvertently disclose these credentials if your repository becomes public). One way to deal with this issue is to put your credential in an external file that is conditionally read:

1. add a conditional to your database configuration code that…
2. checks if an environment variable named NODE_ENV is set to PRODUCTION
   • read the excellent digital ocean summary regarding environment variables
   • use [process.env.NAME_OF_VARIABLE] to access environment variables through node
3. if the above is true, then read a file synchronously (blocking) by using fs.readFileSync
4. the file that is read in will be a json file that's not in version control… that contains the database connection string for your application when deployed on the server

(Note that this is a simple way of managing configuration, so if you plan on actually deploying your app outside of this class, consider using a configuration / configuration management library, like node-convict)

To add an external configuration file, follow these steps →

1. add config.json to your .gitignore so that your credentials don't inadvertently get committed
2. in db.js add the following code before mongoose.connect:

   // is the environment variable, NODE_ENV, set to PRODUCTION? 
   let dbconf;
   if (process.env.NODE_ENV === 'PRODUCTION') {
    // if we're in PRODUCTION mode, then read the configration from a file
    // use blocking file io to do this...
    const fs = require('fs');
    const path = require('path');
    const fn = path.join(__dirname, 'config.json');
    const data = fs.readFileSync(fn);
   
    // our configuration file will be in json, so parse it and set the
    // conenction string appropriately!
    const conf = JSON.parse(data);
    dbconf = conf.dbconf;
   } else {
    // if we're not in PRODUCTION mode, then use
    dbconf = 'mongodb://localhost/YOUR_DATABASE_NAME_HERE';
   }
   

3. when you use mongoose.connect, pass in dbconf as the argument instead of a hardcoded string
   • mongoose.connect(dbconf);
4. you can test that everything works by:
   • try running your application locally (on your own computer) without any environment variables (just use node app.js or ./bin/www)
     • … and inserting data
     • via your form
     • (make sure the data persists)
   • create a config.json that contains a connection string with a different database name (change the last part, after localhost, to a different database name, like mongodb://localhost/finalprojectconfig)
     • the key should be "dbconf" (remember that json keys are double quoted)
     • the value should be "mongodb://localhost/SOME_OTHER_DATABASE_NAME"
     • `{"dbconf":"mongodb://localhost/SOME_OTHER_DATABASE_NAME"}
   • then, run your application again, this time forcing your app to use the config file
     • NODE_ENV=PRODUCTION node app.js or NODE_ENV=PRODUCTION ./bin/www
   • the new database shouldn't have any data that you entered previously!
   • DO NOT COMMIT config.json (in fact, it should be in your .gitignore as the previous instructions specify)
   • (you'll create a config.json on the server)
5. Finally, because you'll want to use a custom port number, you'll have to modify your app.listen call so that it uses an environment variable
   • if you're using express generator, then this is already done for you
   • otherwise, change your call to app.listen so that it checks for an environment variable, PORT, first… otherwise, default to 300
   • app.listen(process.env.PORT || 3000);
6. commit and push your code

1. make sure you're logged in to you remote server (linserv1 or linserv2) via ssh (remember, you have to ssh to access.cims.nyu.edu first, then ssh to your server)

2. Clone your repository into your opt directory (you can find your full repository name on GitHub). Remember to substitute REPOSITORY_NAME with your actual repository name:

cd ~/opt
git clone https://github.com/nyu-csci-ua-0480-SECTION-SEMESTER-YEAR/REPOSITORY_NAME

1. Alternatively, you can use sftp or scp to transfer files to the server

Before installing your project's dependencies, make sure you're using the right version of node. and that you're logged in to the right server - either linserv1 or linserv2.

1. check your node version with node -v
2. it should be grader than 6.x.x
3. if it isn't, make sure that you installed the latest node with nvm from the beginning of these instructions
4. if you already installed, then try running your installed nvm: . ~/.nvm/nvm.sh

Just like local development, you'll have to install your projects dependencies. Replace REPOSITORY_NAME with your project's actual repository name in the following command.

cd ~/opt/REPOSITORY_NAME/ && npm install

(basically… just go to the same directory that your package.json is in… and install from there.

Create a config.js file on the server to add the PRODUCTION version of your mongodb connection string. You can use a commandline text editor, like nano, vim or emacs. We'll use nano in these examples (but you can use vim or emacs as well).

Again, make sure you're logged in to you remote server (linserv1 or linserv2) via ssh (remember, you have to ssh to access.cims.nyu.edu first, then ssh to your server). Do not run these commands if your are only logged in to cims.

1. In your project directory, create and open your file by:
   

   nano config.json

2. Then add your connection string so that it includes username and password (that you retrieved above):
   

   {"dbconf":"mongodb://jversoza:my_password@class-mongodb.cims.nyu.edu/jversoza"}

3. Remember, the name of your database is the same as your username!
4. Save it by CONTROL+O to write out the file. Press RETURN/ENTER to accept the file name.
5. Quit nano by CONTROL+X
6. Test your application. Substitute APP_PORT_NUMBER with the port number you retrieved from Part 1 and add the environment variable, NODE_ENV.
   • Run bin/www or… if you didn't use express generator, use node app.js. Don't use nodemon to run it.
     

     PORT=APP_PORT_NUMBER NODE_ENV=PRODUCTION bin/www

   • If it starts up fine, try connecting to it from your browser:
     

   http://server_name:APP_PORT_NUMBER/

7. Troubleshooting:
   • If you see the following error
     

   /home/net_id/opt/final-project/node_modules/mongoose/node_modules/mongodb/lib/server.js:235
        process.nextTick(function() { throw err; })
   Error: connect ECONNREFUSED
    at errnoException (net.js:905:11)
    at Object.afterConnect [as oncomplete] (net.js:896:19)

   You can't connect to your database; double check your mongoose connection string… and review part 3.

   • If you see Error: listen EADDRINUSE … the port you've been assigned is already in use. Either your application is already running or someone inadvertently is running there application on your port. Contact me if it's the latter.

In order to have your project run even when you're not logged in to the server through an interactive shell, you'll have to run it as a daemon. We'll use a node module called forever to do this. It will run your application in the background and give you tools to manage it (start and stop).

1. Install forever:

cd ~/usr/local/lib/
npm install forever

1. Run your app with forever start. Substitute APP_PORT_NUMBER with your actual port number. Note the -o and -e options; they specify where your application's output (debug and error output) should go. The first version is with app.js (without express generator), and the second is with bin/www

cd ~/opt/final-project/
export PORT=APP_PORT_NUMBER; export NODE_ENV=PRODUCTION; ~/usr/local/lib/node_modules/.bin/forever -o ~/var/log/app.log -e ~/var/log/app_error.log start app.js

cd ~/opt/final-project/
export PORT=APP_PORT_NUMBER; export NODE_ENV=PRODUCTION; ~/usr/local/lib/node_modules/.bin/forever -o ~/var/log/app.log -e ~/var/log/app_error.log start bin/www

1. Check that everything started up fine by looking for the process id, and then checking the log files.

ps aux | grep forever | grep -v grep

Your application would usually log debug lines and errors to the console. However, since you're running your app as a daemon, all of that output is now dumped into log files, which are in ~/var/log. To view the last few lines of the error and app log:

tail ~/var/log/app.log ~/var/log/app_error.log

1. Listing managed apps and Stopping forever / shutting down your app:

# show managed apps
~/usr/local/lib/node_modules/.bin/forever list
# stop all apps
~/usr/local/lib/node_modules/.bin/forever stopall

1. Viewing error logs and debug statements.

For all logs:

cat ~/var/log/app.log ~/var/log/app_error.log

For just the last few lines:

tail ~/var/log/app.log ~/var/log/app_error.log

1. Viewing error logs and debug statements as they happen use the -f flag with tail. This is super useful for debugging.

tail ~/var/log/app.log ~/var/log/app_error.log

1. (Optional) Want to have forever accessible in your path? Add PATH=$PATH:home/jjv222/usr/local/lib/node_modules/.bin/ to your .bash_profile

1. If you'd like to reinstall your application (again, with REPOSITORY_NAME replaced with your repository name):

2. rm -rf ~/opt/REPOSITORY_NAME

3. go back to Part 5

4. If you'd like to simply redeploy or update your application, run git pull to update your code on the server:
   1. cd ~/opt/REPOSITORY_NAME
   2. git pull