last-modified/if-modified-since

[johndoe444]

Dear friends,

I have this code which attempts to get the last modified date of an http resource:

request = urllib2.Request('http://www.imdb.com')
>>> opener = urllib2.build_opener()
>>> request.headers
{}
>>> first = opener.open(request)
>>> first.headers.dict
{'transfer-encoding': 'chunked', 'set-cookie': 'session-id=209.251.138.146.1246519635845007; path=/; domain=.imdb.com, uu=bN4RO5T1YWqLBgRAupwcdgCRSyzj0j26pvJ+qYPCXqmT117Zs9L+/LfGO4zGx50bwJFtjNPSXuqj8g4pgDJ+meCRbaoGcS2aoNZb7PiBbbqjkm66I/I+qSAyfpnj8l7qA/I+qTAyfpngkW2KBnEtmqDXe20AkW26oLFtuqC3KyrjwW26o5IOuZOyLtnAgc0ZsLFtuqCRbergwR2Ko=;expires=Thu, 30 Dec 2037 00:00:00 GMT;path=/;domain=.imdb.com, cs=xOA2Pyx3xsSbt7wMU8ZOhAbGfbqgkW2NmIHl2qPyXuoj8j6pMDJ+meCRbYoGES2aoJFb/feFHbqjsg7/p9HNySCRWyxAGW26oKdbraCRbbqgsW26oJFt+uDBHYqg==;expires=Fri, 03 Jul 2009 07:00:00 GMT;path=/;domain=.imdb.com', 'vary': 'Accept-Encoding,User-Agent', 'server': 'Server', 'connection': 'close', 'cache-control': 'private', 'date': 'Thu, 02 Jul 2009 07:27:15 GMT', 'p3p': 'policyref="http://i.imdb.com/images/p3p.xml",CP="IMDB "', 'content-type': 'text/html'}

As you see in the server's response last-modified header is not set. In this circumstances, how to force the server to give data only if the cached page at user has been modified by the server instead of the server making every response refreshed.

Thanks

[siddhant3s]

>As you see in the server's response last-modified header is not set. In this
>circumstances, how to force the server to give data only if the cached page at
>user has been modified by the server instead of the server making every
>response refreshed.
Not all server support this facility. This server does, but not for the main page:
Below I am listing the conversation between my browser (firefox) and the website:

GET / HTTP/1.1

Host: www.imdb.com

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061015 Firefox/3.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

Cookie: uu=CX+yLlMGer760ThYKb3rwgcx2n6Ud0w+kURafREkT33RASpdsSQKLdBEfx3x5+1Zxxd6SMe3TD6UZa8dtGfsbeQ0zt3xYcyex0AquAXx2t4xMVrY0DG6KNBRGvi2MVqptzPY3jczCP4z08iuN7cYesX0Xz219G89FfRfXafkXx20hD8eNAQvLbe0Xx3HF0yeMfcMHpdQemjPB0w+lDRPPhR0f33HtF8d1IQ/HjR0Dx0XtF8d1xdMDjH3DB6XUVrpNDdMPpQkb23kNA9d1wfsnYc3TD6XMQqu1DdMPpQEHw3kNA9dxwfsnYc3TD6XF0xu10c8Dp



HTTP/1.x 200 OK

Date: Thu, 02 Jul 2009 08:04:34 GMT

Server: Server

Cache-Control: private

Content-Type: text/html

Set-Cookie: session-id=122.162.244.5.1246521874392211; path=/; domain=.imdb.com

Set-Cookie: uu=dwqwj+qqsAa9zXeKESJ1rwcx2n6Ud0w+kURafREkT33RASpdsSQKLdBEfx3x5+1Zxxd6SMe3TD6UZa8dtGfsbeQ0zt3xYcyex0AquAXx2t4xMVrY0DG6KNBRGvi2MVqptzPY3jczCP4z08iuN7cYesX0Xz219G89FfRfXafkXx20hD8eNAQvLbe0Xx3HF0yeMYcMHpcXTBjUND9+lHQvPRe0Xx3XF0wuMfcMHpdRWuk0V0w+lDQffeQ0D13HB+ydhzdMPpFBGt7XF0x5pBfEHpQEzz2UNA9dxDfsnYQ0z63kNA9d1yfsnYc3TD6XF0xu10c8Dp;expires=Thu, 30 Dec 2037 00:00:00 GMT;path=/;domain=.imdb.com

Set-Cookie: cs=PpPGYbzrkdWZaIY9FQ1IpwiOAiSO2RITtsmaRI1aYWQNukEHnnoBB87ZEhQoWVIEjtkkY9pOkiSIjCE3Hc/BF7mZspfO2SSyblESJI7vJDOO2RIkjvkSJI7ZEmTOiWIUg=;expires=Fri, 03 Jul 2009 07:00:00 GMT;path=/;domain=.imdb.com

Set-Cookie: session-id=769-6521874-9398196;path=/;domain=.imdb.com

Set-Cookie: session-id-time=1246608274;path=/;domain=.imdb.com

Vary: Accept-Encoding,User-Agent

Content-Encoding: gzip

P3P: policyref="http://i.imdb.com/images/p3p.xml",CP="IMDB "

Content-Length: 9819

nnCoection: close

----------------------------------------------------------

http://i.media-imdb.com/images/SFbda806c2716d56596760a49da9112b27/css2/consumersite.css



GET /images/SFbda806c2716d56596760a49da9112b27/css2/consumersite.css HTTP/1.1

Host: i.media-imdb.com

User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061015 Firefox/3.0

Accept: text/css,*/*;q=0.1

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Connection: keep-alive

Referer: http://www.imdb.com/



HTTP/1.x 200 OK

Server: Server

Last-Modified: Wed, 24 Jun 2009 07:35:41 GMT

Etag: "4295-4a41d74d"

Accept-Ranges: bytes

Content-Type: text/css

P3P: policyref="http://i.imdb.com/images/p3p.xml",CP="IMDB "

nnCoection: close

Vary: Accept-Encoding

Content-Encoding: gzip

Cache-Control: max-age=314712038

Expires: Sat, 22 Jun 2019 20:05:13 GMT

Date: Thu, 02 Jul 2009 08:04:35 GMT

Content-Length: 3673

Connection: keep-alive

--snipped--

As you can see, the main page do not have a e-tag but the css file do have it.