Creating a simple REST application

[Violet_82]

Hi guys,
I wonder if this is something I could do on this forum.
I've never worked with REST, so I thought that it could be a good exercise to try to develop a very small and simple Java application and try to make use of REST as well.
I found a brief which seems simple enough, I'll share it with you (I'm happy to change this of course if you think it's useful as for me it's just practice).

As a Rest Client
I want to submit new orders for bricks
So I can start customers’ orders

Given
A customer wants to buy any number of bricks
When 
A "Create Order" request for a number of bricks is submitted
Then
An Order reference is returned
And
The Order reference is unique to the submission

There will be more stuff later about retrieving an Order, but I thought this is a good start.

Now, I said above that I have no experience with REST, so I've been doing a bit of research and before I start coding I thought it might be a good idea to sort of clarify how we should proceed and the tech needed (I could also upload everything to git so that anybody else interested in it could benefit).

I guess I could start with a normal Java project, have a POJO Order class and a Brick one and a main class which would create the necessary objects, they seem to be the very minimum to start with.
As to how to proceed after that, I might need a bit of a guidance – that's providing that I started the right way of course.

In terms of storage, what's the easiest, using an SQL database of some Spring inbuilt database if there is any such a thing – I'd like to simplify the storage as much as possible so that I can concentrate more on the java code.

Is it a good idea to use Spring boot?
I know this is gonna be a very long thread, but I think it might be useful.

[JamesCherrill]

That seems backewards to me. For a first learning exercise in REST maybe it would be better to start with the essential univeral REST components - a Client and a Server. They should implement the relevant commands (GET etc). You could start with just one resource that the server has a hard-coded test value for. Goal 1 is that the client can send commands to the server, the server parse them, the server respond (JSON, XML?), and the client process the responses (including error codes).
Then I would move on to creating some more challenging data (Brick, Customer) and plugging that in to your working REST infrastructure.

For learning I would avoid existing solutions. First do it yourself so you really undertand the protocol and the issues. Only then will you know how to evaluate alternative solutions and configure them.

Fi=nally, I think any discussion of what database technology to use is not going to help you learn about REST. Ignore it for now.

[Violet_82]

Fair enough. I only suggested Java as that's the language I know, but I see your point. Do you suggest any specific guide/tutorial that you know could help at all?

[JamesCherrill]

Java is a perfectly sensible language for this - no problem. I was suggesting that you code it yourself (in the language of ypur choice) rather than starting with any of the pre-written tools out there. That's how you learn what it's really all about.
The REST protocols are well documented on the web, eg this one, so just look it up and get stuck into some coding.

[Violet_82]

HI thanks. I gave that a good read, I think I have an idea of how it works now. However, I'm not sure where to start from. I mean, I saw this one here https://spring.io/guides/gs/rest-service/ which is a start and does a lot of the configuration for you essentially - but I'm mindful of your suggestion not to go for an already developed solution, but surely in the real world all the configuration and setting up is probably done for you - for example I wouldn't really know how to start coding for this one...

[JamesCherrill]

In the real world you would probably use somethinhg like Spring to implement a real application. But to work with Spring (etc) you need a reasonable understanding of what it's doing for you and why you need to configure the things you need to configure. That's why I recommend starting by coding a simple case yourself.
Where to start? Nothing too big or hard:

Client and Server classes. Instatiate one of each, Give the Client a ref to the Server. In the Client construct a complete correctly formatted GET request and pass it to the Server. In the Server parse the request and return an arbitrary value correctly formtted (eg JSON). In the client parse the returned message and print the value. (For starters I would assume a resource called "Dummy" with an id of 1 and a price of 9.99. I'd hard code those into the Server.)
That will demonstrate that you have understood all the essentials of how REST messages and resource states are created, formatted, parsed.
Enhance it to do a PUT.
Create a more interesting Resource class and enhance your code to GET/PUT that. That will demonstrate that you have understood some of the issues around how REST servers and clients interact with POJOs.

Now look at things like Spring to see how they can simplify your development process in real life - at least you will understand what they are doing for you.

[Violet_82]

OK great, thanks, I will give it a go and post the code here then.
When you mentioned Client and Server I didn't realize you were talking just about normal java classes, that's why I was a bit confused as to where to start from, but from what I can see you mean just simple java classes, that certainly gives me something to start with :-)!

[JamesCherrill]

Yes.
If you are learning about REST messages then it doesn't matter how the messages are sent and received, so passing them in ordinary method calls is the simplest way to do it.
Later on you could plug your working code into a TCP/IP client and server and send the same messages that way. It would be fun, but it wouldn't add much to your undertstanding of REST.

[Violet_82]

OK, so I've started up with something, but I'm not entirely sure that the GET request is right.
Here's what I have so far:

public class TestRest {

    private static Server server;
    private static Client client;
    public static void main(String[] args) {
        server = new Server("Dummy", 1, 9.99);
        client = new Client(server);

    }

}

The Server class

public class Server {   
    private String resourceName;
    private int ID;
    private double price;

    public Server(String resourceName, int iD, double price) {
        this.resourceName = resourceName;
        ID = iD;
        this.price = price;
    }

    public String getResourceName() {
        return resourceName;
    }
    public void setResourceName(String resourceName) {
        this.resourceName = resourceName;
    }
    public int getID() {
        return ID;
    }
    public void setID(int iD) {
        ID = iD;
    }
    public double getPrice() {
        return price;
    }
    public void setPrice(double price) {
        this.price = price;
    }

    public String parseRequest() {
        //TODO:find out how to parse request
        String message = null;      
        return message;
    }

}

The client class

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class Client {

    Server server;

    public Client(Server server) {
        this.server = server;
    }

    public Server getServer() {
        return server;
    }

    public void setServer(Server server) {
        this.server = server;
    }

    public void contructGetRequest() throws Exception {
        String USER_AGENT = "Mozilla/5.0";
        String url = "http://www.google.com";
        URL obj = new URL(url);
        HttpURLConnection con = (HttpURLConnection) obj.openConnection();

        // optional default is GET
        con.setRequestMethod("GET");

        //add request header
        con.setRequestProperty("User-Agent", USER_AGENT);

        int responseCode = con.getResponseCode();
        System.out.println("\nSending 'GET' request to URL : " + url);
        System.out.println("Response Code : " + responseCode);

        BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
        String inputLine;
        StringBuffer response = new StringBuffer();

        while ((inputLine = in.readLine()) != null) {
            response.append(inputLine);
        }
        in.close();

        //print result
        System.out.println(response.toString());

    }

}

[JamesCherrill]

OK so far (despite a load of unneeded getters and setters!) (see below)
Because you will be passing the messages by simple method calls you do not need any http classes. Just construct the request string and pass it to the server

(below) It's a mistake to automatically create getters and setters for everything. It's the opposite of encapsulation. What you should do is make everything private then create getters or setters for only those things that absolutely need to be read/written from outside. Be especially careful in creating setters... do you really want to change that value in the middle of a run?

[Violet_82]

Thanks for the feedback. Fair comment on setters and getters, I have the tendency to add them all the time, I will remove the ones that are not needed.

Because you will be passing the messages by simple method calls you do not need any http classes. Just construct the request string and pass it to the server

I might have misunderstood what you said earlier but I thought that this was the way to construct the request

 String USER_AGENT = "Mozilla/5.0";
        String url = "http://www.google.com";
        URL obj = new URL(url);
        HttpURLConnection con = (HttpURLConnection) obj.openConnection();
        // optional default is GET
        con.setRequestMethod("GET");
        //add request header
        con.setRequestProperty("User-Agent", USER_AGENT);
        int responseCode = con.getResponseCode();
        System.out.println("\nSending 'GET' request to URL : " + url);
        System.out.println("Response Code : " + responseCode);
        BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
        String inputLine;
        StringBuffer response = new StringBuffer();
        while ((inputLine = in.readLine()) != null) {
            response.append(inputLine);
        }
        in.close();
        //print result
        System.out.println(response.toString());
    }

So if I understand correctly you're saying I only need something like this

public void contructGetRequest() throws Exception {
        String USER_AGENT = "Mozilla/5.0";
        String url = "http://www.google.com";
        server.passRequestToServer(USER_AGENT, url);
    }

and the server will have that method which will return the Dummy, 1, 9.99 formatted as json?

[JamesCherrill]

That's going in the right direction. Now you need to do the REST formating and encoding so that all you pass to the server is a single string and all it returns is a single string.

String thisIsWhatWeWouldNormallySendViaTCPIP = (now construct a REST request from scratch)
String thisIsWhatWeWouldGetBack = server.process(thisIsWhatWeWouldNormallySendViaTCPIP);
(now decode and parse thisIsWhatWeWouldGetBack)

HttpURLConnection does a lot of the encoding/formatting for you - which is great. But if you want to understand what' going on you need to dive down and do that yourself. (Once again - I'm responding to a need to learn about REST. If all you want is the shortest way to get something working then I would give different answers)

[Violet_82]

No that's fine, I'd like to try to learn that, but I'm not too sure what a properly formatted request should be. The link you sent a while back essentially says that a GET request would be something like this:

GET http://fashionboutique.com/customers
    Accept: application/json

So I guess this should be my "GET request from scratch"? If that's the case I could combine them in a string and pass them on onto the server as you suggested. Then the server would return back the dummy string, in a string as you've shown in your reply, so something likeso literally Dummy, 1, 9.99. When you then say that I need to decode and parse the string, you mean converting that return string to a json object (JSONObject)?

[JamesCherrill]

Yes. The URL can be simple if the context is known, eg GET CLIENT/123
Don't forget that the URL part should be UTF-8 encoded.
If you accept application/json then it's the server's responsibility to get the data and encode it according to the JSON rules. That results in a single String that the server returns to the client. The client then has to parse that string to extract the values and create/populate a POJO.

What will rapidly become apparent is that the REST message protocol is pretty simple. Encoding/decoding JSON or XML is harder, and managing the objects (both at server and client) is the hardest part. But it's also the most fun.

[Violet_82]

OK, so I refactored the code a little.
Testing the class

public class TestRest {

    private static Server server;
    private static Client client;
    public static void main(String[] args) throws Exception {
        //server = new Server("Dummy", 1, 9.99);
        server = new Server();
        client = new Client(server);
        String getRequest = client.contructGetRequest();
        System.out.println(getRequest);
        String response = server.getResponse();
        System.out.println(response);
    }

}

CLient

import static java.nio.charset.StandardCharsets.*;
public class Client {

    Server server;

    public Client(Server server) {
        this.server = server;
    }

    public Server getServer() {
        return server;
    }

    public void setServer(Server server) {
        this.server = server;
    }

    public String contructGetRequest() throws Exception {

        String request = "GET CLIENT/123";
        byte[] ptext = request.getBytes(ISO_8859_1); 
        return new String(ptext, UTF_8); 

    }

}

Server

public class Server {   
    private final String resourceName = "Dummy";
    private final int ID = 1;
    private final double price = 9.99;

    public String getResourceName() {
        return resourceName;
    }

    public int getID() {
        return ID;
    }

    public double getPrice() {
        return price;
    }

    public String getResponse() {
        return new String(getResourceName() + getID() + String.valueOf(getPrice()));        
    }

}

So here I'm just passing the simple string to the server and then getting the response (hardcoded string) as a response. Now, we said I have to stract the values from the string, but if the string is a single string like this Dummy19.99 I'm not sure I will have much luck in separating it and extracting the values as I don't have any separator or anything I could use to split the string - had it been insteal a key value string I might have had more luck I guess?

[JamesCherrill]

Your request string is incomplete - see the examples linked earlier.
The URL should be UTF-8 encoded and % encoding all non alphanumerics
The returned value must be in a format that the request specifies in one or more accept lines - eg JSON or XML

[Violet_82]

Your request string is incomplete - see the examples linked earlier.

The URL should be UTF-8 encoded and % encoding all non alphanumerics

Sure sorry. It should be:

public String constructGetRequest() throws Exception {

        String request = "GET CLIENT/123 Accept:application/json";
        byte[] ptext = request.getBytes(ISO_8859_1); 
        String utfEncoded = new String(ptext, UTF_8);
        String utfAndNonAlphaCharEncoded = encodeNonAlphanumericalChars(utfEncoded);        
        return utfAndNonAlphaCharEncoded;       
    }

    private String encodeNonAlphanumericalChars(String toEncode) {
        String encodedString = toEncode.replaceAll("[^A-Za-z0-9]", "%");
        return encodedString;
    }

This gets the UTF encoded and replaces the non alphanumerical characters to %.
But I have an issue - I think - converting the response in the Server class to Json

private final String resourceName = "Dummy";
private final int ID = 1;
private final double price = 9.99;
...
public String getResponse() {       
    return new String(getResourceName() + getID() + String.valueOf(getPrice()));        
}

So, I think I need to clarify a few things, sorry.
Rather than the Server class returning a string, can't I get it to return the formatted Json rather than format it in the main() class? The thing is, if Server returns a string, it will return something like Dummy19.99 with or without spaces, not sure yet, but then I have to find a mechanism to get it to look like this

{
  "dummy": {
    "id": 1,
    "price": 9.99
  }
}

in the TestRest (main) class. Isn't is better to convert it to Json inside the Server class before it gets returned to the TestRest class?
One more thing: if we want that string to look like the above json (if not how do we want it to look like?), what would be the best way? What I mean is I have to include the names and the values of the member variables (well at least for ID and price, the resource name doesn't need to be there) in the Json, so is there a way, for instance, to grab the server class and turn that into json or do I have to manually create the json string and include the name of the variables in it and their values to get to something like

{
  "dummy": {
    "id": 1,
    "price": 9.99
  }
}

[JamesCherrill]

The server MUST construct the JSON string with names, brackets etc. to return to the client It may delegate it to other classes of course, but it is repsonsible for returning the data in the format specified by the accept. Doing it manually is good for now; later you may want to look for existing products that do it automatically.

The encoding of the URL still isnlt right. You can;t just delete significant charaters like that! See https://en.wikipedia.org/wiki/Percent-encoding etc. Hint: there's a JavaAPI class for that.

[Violet_82]

You can;t just delete significant charaters like that!

No, I can't, sorry, that was stupid.
So, this is what I found and how I dealt with it in the Client:

public String constructGetRequest() throws Exception {
    String request = "GET CLIENT/123 Accept:application/json";
    byte[] ptext = request.getBytes(ISO_8859_1); 
    String utfEncoded = new String(ptext, UTF_8);
    String encodedRequest = URLEncoder.encode(utfEncoded, "UTF-8");
    return encodedRequest;
}

given this input GET CLIENT/123 Accept:application/json I get back GET+CLIENT%2F123+Accept%3Aapplication%2Fjson. Presumably there is a corresponding decode method or something along those lines to decode.

On the Server class this is how I created the json response:

public String getResponse() {

    String formattedJsonResponse = null;
    JSONObject parentJsonObj = new JSONObject();
    JSONObject childJsonObj = new JSONObject();
    try {           
        //this will produce this format {"Dummy":{"price":9.99,"id":1}}
        childJsonObj.put("id", getID());//add object key val
        childJsonObj.put("price", getPrice());//add object key val
        parentJsonObj.put(getResourceName(), childJsonObj);         
        formattedJsonResponse = parentJsonObj.toString();           
    } catch (JSONException e) {         
        e.printStackTrace();
    }       
    return formattedJsonResponse;       
}   

Is that what we expected? Below I include the full classes for completeness and clarity (feel free to remove them if you think they don't add any value to the post - the TestRest.java hasn't changed)

Client.java

import static java.nio.charset.StandardCharsets.ISO_8859_1;
import static java.nio.charset.StandardCharsets.UTF_8;

import java.net.URLEncoder;
public class Client {

    Server server;

    public Client(Server server) {
        this.server = server;
    }

    public Server getServer() {
        return server;
    }

    public void setServer(Server server) {
        this.server = server;
    }

    public String constructGetRequest() throws Exception {

        String request = "GET CLIENT/123 Accept:application/json";
        byte[] ptext = request.getBytes(ISO_8859_1); 
        String utfEncoded = new String(ptext, UTF_8);
        String encodedRequest = URLEncoder.encode(utfEncoded, "UTF-8");
        return encodedRequest;
    }

}

Server.java

import org.json.JSONException;
import org.json.JSONObject;

public class Server {   
    private final String resourceName = "Dummy";
    private final int ID = 1;
    private final double price = 9.99;

    public String getResourceName() {
        return resourceName;
    }

    public int getID() {
        return ID;
    }

    public double getPrice() {
        return price;
    }

    public String getResponse() {

        String formattedJsonResponse = null;
        JSONObject parentJsonObj = new JSONObject();
        JSONObject childJsonObj = new JSONObject();
        try {
            /*this will get this format
             * 
             * {"Dummy":[{"price":9.99,"id":1}]}
             * */           

            //JSONArray array = new JSONArray();
            //item.put("id", ID);//add object key val
            //item.put("price", price);//add object key val
            //array.put(item);
            //jsonObj.put("Dummy", array);          

            //this will produce this format {"Dummy":{"price":9.99,"id":1}}
            childJsonObj.put("id", getID());//add object key val
            childJsonObj.put("price", getPrice());//add object key val
            parentJsonObj.put(getResourceName(), childJsonObj);         
            formattedJsonResponse = parentJsonObj.toString();           
        } catch (JSONException e) {         
            e.printStackTrace();
        }       
        return formattedJsonResponse;       
    }       
}

[JamesCherrill]

The request URL needs to be encoded (it may contain blanks etc), but not the whole request. I believe URLEncoder will deal with the character set, so lines 24,25 are redundant. Build the request from its constituent parts.

JSONObjectBuilder and JSONObject are useful, but I would suggest getting this working with some simple string concatenation first. IMHO Jackson is far more useful when building real systems, but to make the comparison you first need to understand what problems they are solving. All libraries come with their own learning curve that will just make things harder right now.

[Violet_82]

Thanks for the feedback.
Right, so , wait a sec:

The request URL needs to be encoded (it may contain blanks etc), but not the whole request.

So you mean I have to split String request = "GET CLIENT/123 Accept:application/json"; and only encode GET CLIENT/123? What happens to the other bit, it doesn't get encoded and concatenated to the encoded one and passed on as well? That's a bit confusing, surely the server needs the second part of the string but why encode only the first part and not the second one? That's providing I understood what you were saying.

JSONObjectBuilder and JSONObject are useful, but I would suggest getting this working with some simple string concatenation first

Ah, I thought that would be all right :-). No worries, will do that with string concatenation as you suggested then.

[JamesCherrill]

If you look at that complete string it’s all simple characters and can be parsed by splitting on blanks... except for the URL that may contain all kinds of stuff including embedded blanks. By encoding the URL down to simple non-blank characters you can do simple parsing (split on white space) for the whole thing.
I suggest you pass the raw URL to the method that builds the command, eg
String reply = get(“client/“ + searchID);

[Violet_82]

I suggest you pass the raw URL to the method that builds the command

Right so something like this then:

public class TestRest {

    ...
    public static void main(String[] args) throws Exception {
        ...
        String searchId = "123";
        String fullUrl = "CLIENT/" + searchId;
        String getRequest = client.constructGetRequest(fullUrl);
        ...

which then, in the client class

public String constructGetRequest(String url) throws Exception {        
        String encodedRequest = URLEncoder.encode(url, "UTF-8");
        String requestType = "GET";
        String typeAccepted = "Accept:application/json";
        String fullRequest = requestType + encodedRequest + typeAccepted;   
        return fullRequest;
    }

Like this you pass just the URL that then gets parsed first and then the other bits added to it - although I would have to include the white space manually, for example space between GET and CLIENT/ as this now yields GETCLIENT%2F123Accept:application/json

[JamesCherrill]

Obviously there's more than one way to go, but I personally would do something very much like that. I'm thnking about how it would fit into a bigger context. For example: suppose yo have some kind of menu where each choice returns a different data item. All you need is the URL for each item to pass to constructGetRequest and it's done!

[Violet_82]

OK, so I've done it as above but then I added some spaces here and there, so ended up with this in the Client class (the main class is as above):

    public String constructGetRequest(String url) throws Exception {        
        String encodedRequest = URLEncoder.encode(url, "UTF-8");
        String requestType = "GET";
        String typeAccepted = "Accept:application/json";
        String fullRequest = requestType + " " + encodedRequest + " " + typeAccepted;   
        return fullRequest;
    }

which yields GET CLIENT%2F123 Accept:application/json which I guess it means, as you said earlier, that I can - if needed - split the string at the space for example.

For the second point, the json response, I removed the json objects and done with strings as you mentioned, so in the Server the getResponse method looks like this now

public String getResponse() {
    String response = "{\"" + getResourceName() + "\":{\"id\":" + getID() + ",\"" + "price\":" + getPrice() + "}}";
    return response;
}   

And thsi yields the json response {"Dummy":{"id":1,"price":9.99}}

[Violet_82]

Looking good!

Great :-). So where to now? I will be reviewing the logic just to make sure i understand the various pieces, what do we do now? We need to take it a notch up of course, so what do we do now? Should I close this thread as this is the basics of REST and open another one?

[JamesCherrill]

We're a maybe half way through "the basics" so keep going. As an absolute minimum go for:
Client is passed a resource name and an ID (eg "Client:, 123). It constructs a GET and passes it to the Server. The server parses the request, gets the requested data (you can fake that bit), builds a JSON string and returns it to the client. The client parses the JSON and prints the data.

[Violet_82]

Right, thanks, haven't we done all that already?

Client is passed a resource name and an ID (eg "Client:, 123).

That's already done as we're passing the full URL String searchId = "123"; String fullUrl = "CLIENT/" + searchId; to the client;

It constructs a GET and passes it to the Server

That's done too as the client is encoding the URL and then concatenate the other bits to it;

Parsing and getting the request are essentially faked already, then the Json string is build and passed back to the main class which prints everything?

Parsing and

[JamesCherrill]

Don't fake any of the parsing.
Have two different resources and/or ids so you can test the parsing etc properly

[JamesCherrill]

OK, let's have some fun. Here's some code that will create test environment for you to code against..
First here's a couple of data classes (JavaBeans compliant)

class Customer {

    private int id;
    private String name;

    public Customer() {
    }

    public Customer(int id, String name) {
        this.id = id;
        this.name = name;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Customer{" + "id=" + id + ", name=" + name + '}';
    }
}

class Product {

    private int id;
    private String name;

    public Product() {
    }

    public Product(int id, String name) {
        this.id = id;
        this.name = name;
    }

    public int getId() {
        return id;
    }

    public void setId(int id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "Product{" + "id=" + id + ", name=" + name + '}';
    }

(you'll need those in both the client and the server.)

Now some test code for the client...

    void test() {
        System.out.println(client.get(Customer.class, 1));
        System.out.println(client.get(Product.class, 2));
        // should print the toString for Customer and Product
    }

... and an object database (!) that the server can use to retrieve resources that the client requests...

    Object getResource(String type, int id) {
        switch (type) {
            case "Customer":
                return new Customer(id, "Customer " + id);
            case "Product":
                return new Product(id, "Product " + id);
        }
        return null;
    }

Your mission (if you chose to accept it) is to fill in the gap between the test method and the object databse using REST and JSON