meta refresh and expires

Ok, Lazyweb, earn your keep.

In the Gronk UI, the playlist page reloads every 30 seconds. However, all of the inlined images on it reload too, and that's annoying. I want the HTML to reload (with If-Modified-Since), but I want the browser to realize that it doesn't have to hit the server at all about the images.

I have attempted to set the HTML to expire "now" and the GIFs to expire "in a week". But Mozilla and Firefox reload the GIFs every time they reload the HTML: every 30 seconds, half a dozen HTTP requests are made instead of just one, and sometimes it causes visible flicker on the images.

Is there really, still, no way to do this?

My Apache .htacess file says: <LJ-CUT text=" --More--(34%) ">

    <Files playlist.html>
      ExpiresActive On
      ExpiresByType text/html A0
    <Files *.gif>
      ExpiresActive On
      ExpiresByType image/gif A604800

Yielding the headers:

    % wget -qsO- http://.../playlist.html

      HTTP/1.1 200 OK
      Date: Thu, 05 May 2005 22:09:23 GMT
      Server: Apache
      Last-Modified: Thu, 05 May 2005 22:07:11 GMT
      ETag: "1beac-30ca-31fd21c0"
      Accept-Ranges: bytes
      Content-Length: 12490
      Cache-Control: max-age=0
      Expires: Thu, 05 May 2005 22:09:23 GMT
      Connection: close
      Content-Type: text/html; charset=iso-8859-1

      <META HTTP-EQUIV="Refresh" CONTENT="30">
    % wget -qsO- http://.../up.gif

      HTTP/1.1 200 OK
      Date: Thu, 05 May 2005 22:09:23 GMT
      Server: Apache
      Last-Modified: Thu, 06 Jan 2000 05:29:16 GMT
      ETag: "6b982-56-69f96300"
      Accept-Ranges: bytes
      Content-Length: 86
      Cache-Control: max-age=604800
      Expires: Thu, 12 May 2005 22:09:23 GMT
      Connection: close
      Content-Type: image/gif

      GIF89a ...

Please don't suggest that I rewrite it all using XMLHttpRequest or Java or Flash or whatever. Obviously if I were willing to do that, I'd have done it already.

Tags: , ,
Current Music: Red Aunts -- Hate ♬

48 Responses:

  1. brad says:

    The ghetto fuck-you-browser way I know to do this is to load the new HTML with JavaScript, and have JavaScript also replace the entire body content with the new version.

    And you could put the typical refresh mode in a [noscript] or document.write it out in your head if they don't have a modern browser with XMLHttpRequest.

    I'll hack up a demo and put it on for you to steal.

    • fo0bar says:

      What the hell do you know about making a web site?

      Damn LJ newbies...

    • brad says:

      Please don't suggest that I rewrite it all using XMLHttpRequest or Java or Flash or whatever. Obviously if I were willing to do that, I'd have done it already.

      I could've sworn you added that after my comment. :-)

    • evan says:

      I have this same problem when I reload my LJ friends page -- all of the images people have posted reload.

  2. You amuse the hell out of me. You don't want to use the technologies that came to the fore to deal with the exact problem they were designed to deal with. Hehehehe. Seeing as tho Moz and FF are your target platform, there is a Moz specific technology to deal with this sort of thing, but i suspect you will hate it more than XMLHttpRequest. It involves RDF.

    *evil grin*

  3. tkil says:

    Short answer: I can't think of a way to make the one-page / one-document version work the way you want it to.

    And given the expirations on images vs. html, is the browser really re-loading the images [status 200] from the server, or is it getting 304s as expected, and the flicker is just the reflow of the HTML? If this is the case, I don't see how you can solve it with pure HTML, and some sort of client-side smarts is required.

    If you'd be ok with using javascript (and optionally frames), you can use one frame (or, as <lj user="brad" /> indicates, just javascript) to check for new content ... and only reload the main window when there is new content.

    (I've thought about this w.r.t. SlimServer; its web front-end has the same problem of balancing refreshes vs. flicker.)

  4. brad says:

    Working example, with javascript library. Falls back to old behavior.

  5. sol3 says:

    It looks like firefox is still insisting on doing a get with an "If-modified since" header, even when cached. I believe Expires/Cache-Control really on affect proxies/caches, and are not necessarily obeyed by browsers. Check the access log, and see if apache is at least returning 304's for the image requests. While that doesn't solve the issue of the multiple requests, it should at least keep the images from being shot over the wire repeatedly.

    The image flicker could be less from images reloading and more from page redrawing - stupid question, are you explicitly sizing images (and other elements) - in css or otherwise?

    (This is from digging at my own server and its logs to try to solve a similar issue)

    • jwz says:

      The server is sending 200, not 304. All the IMG tags have width/height.

      • sol3 says:

        Do you know/can you check if firefox is sending the if-modified-since header in its request? This is firefox 1.0 on a mac and apache 2.0.52. (Short of something like brad's javascript above I haven't found a way to force the browser to not even do the if-modified-since header'd get).

        • jwz says:

          I assume it's not sending I-M-S, or else Apache would be sending 304 back (but I don't know an easy way to sniff that traffic). You can see above that the date on the file is in 2000.

          • otterley says:

            I don't know an easy way to sniff that traffic

            You want this.

            • jwz says:

              I have that, but that doesn't mean I believe it. It says 200 and doesn't mention I-M-S, but does that refer to the most-recent request, or the request that put the document in the cache? It's not obvious.

              • at400723 says:

                With Apache 1.3.31 onwards mod_log_forensic can log the headers provided by the browser to a file. Annoyingly they're logged to a separate file from the access_log. Apache provides a new opaque token which can be logged into the access_log to cross-reference the access_log and forensic_log. The best that can be said for this is that it's an accurate way to see what headers get sent, but something a bit easier to use would have been nice. Still, msun't grumble.

                There's a brief article about it here:

              • dzm6 says:

                If you're not trusting LiveHTTPHeaders, you might try SSLTap from NSS tools. It's a simple lightweight passthrough proxy that logs (or echos to stdout()) all the protocol traffic that goes through it.

              • gwillen says:

                tcpdump says Firefox is not sending I-M-S when asking for the image that should be cached. Actually if I have it "Refresh: 5; url=some-other-page-with-the-same-image.htm" instead of refreshing to the same page, it won't even hit the server at all for the cached image. Unfortunately this has the problem described below, that even if it's really the same page, it will forcibly scroll to the top, which is irritating.

          • dachte says:

            Install this Firefox extension to see the headers.

          • sol3 says:

            While digging around I found a comment along these lines:

            - The header If-Modified-Since is discarded if the given date expresses a date in the future.

            Again, passing along possible issues to look at - Apache 2.0.50 doesn't seem to be doing this (my laptop's clock is about 10 minutes ahead of my server's) - but other versions may have (1.x perhaps).

            Though - even if apache was still sending back 304's, that's only half of your issue - firefox'n'friends are still making those requests to begin with.

          • tcpflow is a great way to find out what the real traffic looks like.

          • netsharc says:

            You mentioned before that you use Privoxy. Adding "debug 8" in the config file (under "section 3.1") makes Privoxy log the request and response headers it gets, that's one not-so-difficult way of tracking the requests.

            Assuming livehttpheaders is accurate, perhaps the mis-configuration is on the browser side?

            Also, another trick that the HTTP-implementators use to aid caching is the ETag, it's a hash of the content calculated by the server and sent to the browser. The browser can request the same file, it tells the server the ETag of its local copy (which it has from a previous response), and if they're the same, the server *should* send a 304.

          • edm says:

            It's typically easiest to sniff the HTTP session(s) off the wire (assuming it's not HTTPS and therefore encrypted). ngrep ( can be quite useful with text-based protocols, and ethereal ( has good protocol decoding (eg, try Tools->Follow TCP Stream).

            You might also consider configuring Apache to log the headers that it receives with the requests to verify that the browswer isn't sending I-M-S. (See generally


  6. I managed to trick firefox 1.0.3 by using two pages, a.html and b.html, that have their refresh set to each other. the server logs show 304 for both the pages and the images in them. it doesn't work if a.html refreshes to itself, it has to be two pages.

    • oh, the caching also takes effect. on most of the refreshes it's just probing for the html pages, it doesn't ask about the images until the magical cache says so.

      • hm, the downside of making it two pages, on every refresh it seeks back to the top of page. so it's probably not suitable for what you want. oh well.

        • so far, my conclusion is that the forcible refresh is a mozilla bug, and someone should report it. it's arguable that it's a feature, the forcing might be useful for things like simplistic webcam pages, but it's easy to force image refresh in other ways, and it isn't easy to get if-modified refresh any other way, and this can be a huge performance hit for many similar applications. so it's a bug.

          the other workarounds I can think of besides javascript are things like iframes and css hacks, which don't fit your goals either.

          • for the record, iframe and css don't help much. http-equiv=refresh still forces reloading of images mentioned in a stylesheet. images in an iframe aren't reloaded, but there seems to be an unavoidable flicker as the browser re-renders the page and then adds the iframes.

            another possible workaround is to build the images with html rather than gifs. like, convert an image into a table with colored cells as pixels, or do it with css boxes, or unicode characters. that's getting kind of silly.

          • proub says:

            Reported as an RFE in 2000. Doesn't seem to be at the top of anyone's list. Added my vote, anyway.

            • gwillen says:

              Voted for, and commented about the behavior of "Refresh:", since they all appear to be thinking in terms of the UI's Reload button.

    • strspn says:

      I wonder if that would work with filenames a.cgi?noop1 and a.cgi?noop2

  7. spoonyfork says:

    Is AllowOverride All set for the directory in httpd.conf?

  8. mark242 says:

    The problem is in your meta refresh tag. Most browsers interpret this as "completely invalidate all elements on the page and refresh everything." I just duplicated your test case on Safari on my local box.

    My only suggestion is to try this snippet of Javascript in place of your meta refresh tag:

    <script language="JavaScript">



    (adjust the 3000 to taste)

  9. you're like the rat that keeps shocking itself, only you don't even get anything out of it.

  10. lohphat says:

    It doesn't happen in PointCast.

    /runs like hell

  11. cloudblur says:

    What are you cache settings in your browser? Firefox, if set to "Check every time" has the behavior of not caching images and not doing If-Modified-Since requests that normally result in 304.

    It might be your cache settings, what about privoxy maybe stripping things?

    • edge_walker says:

      Here’s a funny: where do you change that setting in Firefox? I just looked through all of the entire Preferences dialog and can’t find it anywhere.

  12. luserspaz says:

    I was trying to find out what the specified behavior ought to be, but I then realized that this is a Netscape invention, so the entirety of the specification is in this document. It's a bit underspecified.

    That HTTP header, in turn, tells the browser to go reload (refresh) this document after 1 second has elapsed.

    It doesn't really say whether the browser should treat it as a forced reload, or what to do about getting images, etc from cache. I'm also willing to bet there are pages that rely on the current behavior.

  13. legolas says:

    Not sure if your requirements also exclude javascript, if not, what would happen if you used a refresh(false) on a page that is expired? Docs say passing false will not reload it form the server, I wonder if this is true even if the page is expired.

    Also, I have seen many suggestions using ping pong between 2 html pages etc. I wonder if explicitly reloading the same url might produce the same effect, i.e. what would this do:
    <META HTTP-EQUIV="refresh" CONTENT="5; URL=http://__link to the same page__">