We have an API :) - Page 6

[Dani]

Works fine for me??

http://www.daniweb.com/api/articles?callback=?

[Dani]

If it works from the direct URL but not from jQuery, here's why:

You need to heed the error message. It says that a cachebuster is detected.

By default, jQuery will add a random query string to the end of AJAX calls to force a fresh version of the page to load, instead of loading from the cache. However, this fakes out our caching system and we detect and prevent it. You will need to pass in the {'cache': false} parameter into jQuery.

Ah! Me sees. Thanks.

[Dani]

Clarification: NOt only does it fake out our own caching system, but it pulls more requests than it needs to. Cache your API requests, darn it! :) It's only polite.

[Dani]

OK I just figured out what the problem is.

I was under the impression that my cache-buster detection would be a non-issue because jQuery sets cache to true by default. Therefore, if someone was forcing a cache-buster on my API, that's a bit rude :)

However, I just read this in the jQuery documentation:

cache (default: true, false for dataType 'script' and 'jsonp')
If set to false, it will force requested pages not to be cached by the browser. Note: Setting cache to false will only work correctly with HEAD and GET requests. It works by appending "_={timestamp}" to the GET parameters. The parameter is not needed for other types of requests, except in IE8 when a POST is made to a URL that has already been requested by a GET.

Apparently cache busters are enabled by default for jsonp. UGH.

[Dani]

So thank you for that bug report, but now what do I do about it??? When jQuery uses a cache-buster, then two things happen:

1. It forces your web browser to do a hard refresh each time the content is pulled, instead of listening to proper cache/expires headers. This can result in many additional queries to our server.
2. We cache all API requests for up to an hour in Memcached. Requests are cached based on the full URI string passed in, so we want to prevent cachebusters from appending random strings to the URIs, because that bypasses Memcached.

However, I was not aware that cachebusters were enabled by default for jsonp, which is a bit odd IMHO. However, it's a bit of a double edged sword now because I can see this being a sticking point for nearly everyone as our API won't work out of the box.

Opinions?

Still getting the same (at bottom of page):

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js" type="text/javascript"></script>
<script>
$.ajaxSetup({
   cache: false
})

function getJSON(){
    var url = 'http://www.daniweb.com/api/articles?callback=?';
    $.getJSON(url).done(function(json){
        document.write(JSON.stringify(json));       
    });
}

getJSON();
</script>

I'm obviously doing something wrong. Can't figure out what though...

[Dani]

Let me try :) ... Give me 2 minutes.

[Dani]

I copied/pasted the OAuth demo onto a separate domain (to make sure there wasn't any same-origin policy issues going on) and it worked fine.

Still trying to get your script to work. Hmm ...

Doh:

Need to set

cache: true

It then works.

[Dani]

Yeah, I just got it working myself. I copied/pasted your code and completely didn't realize that the point is you DO want to cache. Sorry for the bad first impression dealing with my cache buster detection.

The thing is, more than the resource-savings, it's really a security measure. If you're unauthenticated, then we want to ensure you have no alternative but to pull the same thing for up to an hour, to deter people from writing scripts that will poll the same thing every minute.

And if you are authenticated, then feel free to cachebust all you want, because I know your DaniWeb username, and I know your IP address, and if I feel like you're really abusing the system I can ban you.

[Dani]

And sorry for my bad advice! Just realized it was me that told you to write cache: false.

I'm going to go ahead and try to get my cache-busting detection thing to strip out jQuery's cachebuster, making it ineffective (at least as far as Memcached is concerned).

This is far easier than having to rely on server-side code, well for the NoAuth stuff anyway.

[Dani]

The stuff with auth is even easier because it doesn't impose any caching restraints!! :)

Seriously, try copying and pasting my demo. It should work out of the box.

to deter people from writing scripts that will poll the same thing every minute

What, like my example script, you mean? :)

[Dani]

You know what would be a cute idea? How about writing a bookmarklet that polls for latest posts in articles you're watching or for your new private messages as you browse the web. You'll get an instant alert wherever you are instead of waiting up to an hour for the email notification to come in.

[Mike Askew]

You know what would be a cute idea? How about writing a bookmarklet that polls for latest posts in articles you're watching or for your new private messages as you browse the web. You'll get an instant alert wherever you are instead of waiting up to an hour for the email notification to come in.

That'd be an interesting one for sure.

I'm currently fiddling with a C# interface of the API (hiding the backend requests etc). Having to put the JSON into XML at present and then manipulating it, so unless I find a better way I'm sure some of the more experienced devs will have a better interface if they tried :)

Work is busy at present though so won't get a chance to properly crack on with this mini project of mine until the weekend :(

[Dani]

OK, jQuery's cache buster now gets stripped out by our cachebuster detection system. This means that you don't have to cache your jQuery calls anymore. They will still force a hard refresh to our server, but our server will strip out the jQuery cachebuster parameter when comparing to the URIs it has saved in Memcached, so you still might get stale data.

[Mike Askew]

Dani do you happen to have a map of which forum code is associated to each forum (Before I manually go look at them all)?

Ie. 61 = C#

[riahc3]

But kept getting this:

I noticed this happened with the 'nested' parameter for forums too (just straight url into browser bar) - is this expected?

Ive had this strange bug many times; Depends on how you implement it.

Using ?user=riahc3 doesnt work.

But using other functions does.

I didnt know if was a bug or what.

[riahc3]

OK, jQuery's cache buster now gets stripped out by our cachebuster detection system. This means that you don't have to cache your jQuery calls anymore. They will still force a hard refresh to our server, but our server will strip out the jQuery cachebuster parameter when comparing to the URIs it has saved in Memcached, so you still might get stale data.

It happened to me as well in C# Ill try to look into it and give some feedback.

[Dani]

Dani do you happen to have a map of which forum code is associated to each forum (Before I manually go look at them all)?

Here is a nested hierachy of how all forums relate to each other (along with their ID #s): http://www.daniweb.com/api/forums/descendants?nested=1
It's actually the exact function that we use to build our navigation drop down menu and forum listings.

Using ?user=riahc3 doesnt work.

?user= is not a valid query parameter.

[riahc3]

?user= is not a valid query parameter.

I saw that and I thought to myself "Damn I have to edit it" but I just got too lazy to do it. Sorry about that.

I ment ?username=riahc3

[Dani]

Works fine for me??

http://www.daniweb.com/api/members?username=Dani

[Mike Askew]

Thanks Dani :)

[riahc3]

Works fine for me??

http://www.daniweb.com/api/members?username=Dani

I dont know if you made changes or not but that did not work at the time I started the C# proof of concept, I assure you.

BTW, do you have a log of who accesses it? This was you could help us troubleshoot in case we have a problem developing with the API...

[riahc3]

BTW, what does http://www.daniweb.com/api/members return? It returns some users but is there some kind of condition?

[Dani]

I dont know if you made changes or not but that did not work at the time I started the C# proof of concept, I assure you.

Suuuuuurrrreeeee ...

BTW, do you have a log of who accesses it? This was you could help us troubleshoot in case we have a problem developing with the API...

I don't. I just save the required OAuth information. In the long run, I'm going back and forth between it being good to track malicious behavior and it being just too resource intensive to track and record every page hit.

[riahc3]

BTW, I see that Dani has started to put PHP and Javascript examples; I could try my best to do all of them in C#/Java........Would anyone be intrested? And what language do you perfer? (If you say Java, you deserve to die......slowly)

[Dani]

I could try my best to do all of them in C#/Java

There's really just two things to port: Using OAuth to gain an access token, and fetching something. Once you know how to fetch one thing you can pretty much just change the URI to fetch anything you want.