Responsive Images Meeting Notes

These are my notes about the responsive images meeting in Paris on 10th September 2013. The following are the solution proposals. We will shortly be discussing next steps. These are not the official minutes.

Updated notes to include whole day. In conclusion, DPR is highest priority. Both srcset and picture are viable for last call.

• Mat Marquis had a very good video introduction explaining the history and reasons
• Mark giving a BBC overview
  • 83 million views this month
    • About 57.2 million requests a day (if CDNs caching etc were not being used)
    • Core experience 15 HTTP requests, 55kb
    • "IE8 isn't going anywhere" so they are adding support for it
    • ImageChef - wrapper for Image Magick
    • Using their version of responsive images since 2011 - No support for pixel density - Doesn't follow any proposals
    • srcset pollyfill not a good solution for BBC since it downloads all pictures
    • picture pollyfill not good solution either due to markup bloat
    • Uses placeholders
    • Only used for enhanced (javascript) version
    • www.github.com/BBC-News/Imager.js
    • Lazyload in the sense that core experience is always loaded first and then larger images are downloaded if need be
• Akamai - Used primarily for large sites - Advanced CDN - Smart Image Delivery - Detect device capabilities - Delivery decisions such as which server to get from - Samsung is good at providing properties of device in UA - Image Converter - Image manipulation API - Supports art direction - Still requires superset image since can only downsize - Front-end Optimisation - Less control than BBC since it is constrained due to supporting all websites rather than being customised for one - Uses origin image as basis, retina if possible - Create multiple versions of the image - Rounds up currently resolution to closest version - Prefetch tags where possible- Low quality version of image before full image - Adaptive Image Compression - Every image goes to server so there is always a log of bandwidth latency etc. - Very useful to know which size image is best for connection - Boris Smus said about pollyfills having same problem in that low quality will be changed with high quality, so it is good to hear that users don't mind seeing it low quality first. - Google Data Proxy had the problem that users preferred bandwidth over images loading quickly as low quality then upgrading to high quality
• srcset - Ted O'Connor - Webkit supporting a limited version of srcset, patch coming soon for full support - Gecko not yet supporting but working on it - webkit limited to resolution due to it being considered that a lot of the use cases are outside of responsive images. - 'The issues of art direction and resource allocation can be addressed on a platform level not just responsive images' (slightly paraphrased) - Thinks full syntax for srcset is complicated so keeping simple for now - "picture as currently specified is far far far far far too complex" - Regarding viewport in css "we didn't see the point of adding a redundant feature to css" - Very against art direction use case being part of responsive images solution. Suggests using css. - I mentioned about user generated images, they would not be wanting to write css etc. Reply was basically that the images would not be widely different (quite a generalisation I think). - Question about intrinsic size of images, answer is that webkit does support it.
  • Picture - Marcos Cceres
    • "We have come to the realisation that the only use case we have about this is art direction"
    • Trying to simplify picture so that it isn't a "kitchen sink"
    • Want to use srcset and picture element to get best of both worlds
    • Picture element wanting to add type to support the last use case that srcset does not support such as for webp
    • Very easy to pollyfill, some already exist and are being used on large sites
    • Preload problems with img fallback, but solved using web components or postpone attribute
    • Don't want to accidentally opt-out of preload by pollyfilling wrong
    • type allows you to define MIME type. Important for img but doesn't have it. Client-side fallback for when servers say it supports everything
      • Could be issues if media and srcset specify different sizes
    • Yoav recommends using srcset for resolution switching only and art direction in picture, that way using ems etc won't break layout.
    • Question: Is there a priority of use cases? Answer: Trying to simplify picture element.
    • Question: What does the BBC and other large sites think of the large syntax of picture? Answer: Anything marginally complicated with srcset is convoluted markup. We wanted the simplicity of srcset with more functionality, so anything we can do to strip down the picture element then we will be happy with. That amount of markup throughout the page is a bit worrying. Unmaintainable.
    • From an implementation point of view, srcset is much more predictable.
    • Yoav says regarding preprocessor it is fairly simple. But there are edge cases.
    • Picture element makes interactivity such as slideshows slightly awkward. Involving loops etc.
    • Easier to manipulate picture than srcset
    • Yoav: Content negotiation for everything except content images. Basically the only external resource that doesn't have it.
• Ilya Grigorik on Client Hints - This is just Content Negotiation - "We are not implementing anything new, this is what HTTP was designed to do" - Used for WebP in Chrome - External CDNs already use it - Suggestion is to add CH-DPR to HTTP header of images - Server can select static image or dynamically resize/adapt it - "If we had img then we keep img" - Compatible with srcset to override client hints - Massive benefit over srcset in that it can handle all pixel densities, no need to have separate micro syntax for each. - The majority of servers are able to support it - Multiple headers instead of a list works better with current cache implementations - Needs a way to request/change image based on viewport change.- Scepticism about adding more HTTP headers since there is no way to remove them. For example you have to send a user agent with either IE or Gecko due to user agent sniffing. - Content encoding is specified to show encodings that are supported, it only shows gzip. - 'that is only a indication of gzip's success' - Accept language and others failed, cannot remove. - "with dpr you have about 10 breakpoints" - Client hints require width and height in markup
• Yoav Weiss - New Image Format - A container with multiple layers and an offset table on top - Layers are residual images, including repositioning - Tested with WebP, should work with JPEG-XR - Not JPEG since residual image is too big- Fetching relies on HTTP ranges - Needs caching to support ranges better, no reason why they can't - Fetches an initial range - Browser fetches the rest of image gradually - Can be optimised with a manifest - Markup left untouched - Browser can easily fetch layer needed if orientation changes - Big disadvantage is that there are a lot of layers, will take time to create - Not yet tested decoding performance - HTTP2 parallel image loading very quickly shows low quality images, in comparison to HTTP1.1 show a few in high quality and then not loading the rest (SPDY too) - Ilya - Trade off between bytes and progressive loading - This would be ideal in use with a CMS that allowed layers to be added with a nice UI - We need a solution in the mean time while this is being developed since it will take a long time to create. Can co-exist with picture etc.
• Robin Berjon SVG's Switch element - SVG switch picks a child for rendering, the chosen one is the first to pass a test - Only works in SVG context, but can be very similar to picture element and almost works today - xlink:href - media is not supported in SVG switch - Suggests adding media to SVG switch - Good compatibility except
• Microsoft are interested in learning about all of this and it is about time to start planning for IE12 so perfect timing to decide what to implement. - Microsoft.com already uses pollyfill
• DPR switching is top priority - Mozilla starting to work on srcset - Update: Marcos has clarified this, they are getting somebody to work on it. No one has been assigned yet. - Behind a flag in Chrome until mutual agreement
• Yoav: "srcset is great for resolution switching, for viewport switching it could be improved but the thing that is important is that it can only do one or the other." "viewport is something the browser can override, art direction it cannot" "I would not rely on CSS and postpone"
• Marcos: w and h not implemented since they look like width and height and is confusing
• Need to clarify units, very confusing if not matching CSS units module
• "There are a bunch of stuff with media queries that do not happen that early" "I do not even think it is possible to run them at prefetch" "importing media queries either contain the thing doing the importing or the future of media queries or both"
• Ilya: "Should w and h be removed from the spec? Nobody are arguing for them"
• Yoav: "If we consider srcset just for resolution switching, and we drop the h which I never saw the point of, then we could combine the values of x and w into a single unit. I have a syntax proposal which is probably lacking. But I think they should be combined." "Should we combine them or make a list?" - It assumes the developer knows what happens with the images, which is not ideal.
• Microsoft are thinking about 'plateaus' which are defined in a table - Consolidating multiple resolutions to send the same such as 1.3 and 1.5 get same image. - Marcos says Android already had this.
• Yoav: "The confusion part [srcset syntax] is education. The problem is that the syntax is verbose."
• Ilya: "complexity is because it is a big space, there are dozens of combinations" "it is easier to get the server side right on apache and nginx than getting every developer to get it right on their sites"
• Couple of mentions about url templating (rfc6570). Quickly agreed bad idea
• Boris: "Not sure that people who dislike srcset syntax would use it. They tend to prefer picture so why not use it too?" - Ted: "picture as a new element is a big thing" "It would be better to not solve the problem than solve it in a complex way"
• Bandwidth is still an important use case, despite not having easy access to measuring it. - Yoav: "Much easier to measure bandwidth using SPDY or HTTP2"
• Everyone is asking what w and h actually means today - Similar to max-width etc in css - If implementors don't understand then shows that it will be quite hard getting all developers to understand
• "It shouldn't be possible for some crummy developer to write srcset that nobody else will understand"
• Ilya: "If we drop h then it becomes kind of parsable
• Boris is suggesting an order of size for srcset - Marcos doesn't want it to break if the web dev writes it in an order they prefer
• Yoav: "About 25% of 150 responsive sites already used art-direction, about 50% would have benefited" "Not many clients turn it down when offered"
• Art direction used by Loreal to move text in an image advert from part/next to image to below it.
• Ilya: "Art direction can be done server side"
• A lot of talk in irc about art-direction being important, from front-end developer viewpoint
• Ilya: "is it worth making art-direction a first class feature?" He is suggesting that hacks are continued to be used.
• Xbox and other devices can have multiple windows on different screens showing companion images etc.
• Ted: Both srcset and picture could go to last call - Marcos: picture will simplified (as explained earlier) before last call
• Marcos: "These are not small sites that are hitting this issue [art direction]" "content will be king, it is bubbling up already"
  • General consensus that DPR should be the first thing to tackle.
• "To get everyone in the same room is a massive deal"