Full Text RSS Feeds

2020-04-15 6 min read Behind the scenes Blog improvements Tech Web stuff Teknikal_Domain Unable to load comment count

After reading on Kev Quirk’s blog post about including the full text of a post instead of just an excerpt, I’ve decided to do that here. And I’ll outline the before, after, and (naturally) all the steps in-between that caused much headache (and late-night coffee consumption).

Reasoning

Off of Kev’s reasoning, accessibility. Users can tell their RSS reader to style content to their liking, since the reader itself just pulls raw content, and as such can change things like the contrast, font size, or whatnot.

Off of the reasoning of some people that I know and talk to, it’s a lot less hassle to have to open one app, read part, then open another app just to browse to the post to read the rest, it’s a lot easier to just be able to read the entire thing from the start.

Before

Hugo uses a template, in my case, index.rss.xml, to generate the index.xml RSS feed file. Here’s part of it:

<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>
        {{ if eq  .Title  .Site.Title }}{{ .Site.Title }}{{ else }}{{ with .Title }}{{.}} on {{ end }}{{ .Site.Title }}{{ end }}</title>
    <link>{{ .Permalink }}</link>
    <description>Recent content {{ if ne  .Title  .Site.Title }}{{ with .Title }}in {{.}} {{ end }}{{ end }}on {{ .Site.Title }}</description>
    <generator>Hugo -- gohugo.io</generator>{{ with .Site.LanguageCode }}
    <language>{{.}}</language>{{end}}{{ with .Site.Author.email }}
    <managingEditor>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</managingEditor>{{end}}{{ with .Site.Author.email }}
    <webMaster>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</webMaster>{{end}}{{ with .Site.Copyright }}
    <copyright>{{.}}</copyright>{{end}}{{ if not .Date.IsZero }}
    <lastBuildDate>{{ .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" | safeHTML }}</lastBuildDate>{{ end }}
    {{ with .OutputFormats.Get "RSS" }}
	   {{ printf "<atom:link href=%q rel=\"self\" type=%q />" .Permalink .MediaType | safeHTML }}
    {{ end }}
    {{ range $pages }}
    <item>
      <title>{{ .Title }}</title>
      <link>{{ .Permalink }}</link>
      <pubDate>{{ .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" | safeHTML }}</pubDate>
      {{ with .Site.Author.email }}<author>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</author>{{end}}
      <guid>{{ .Permalink }}</guid>
      <description>{{ .Summary | html }}</description>
    </item>
    {{ end }}
  </channel>
</rss>

line 36 is emitting the <description> tag, which is the content. This is the page’s summary (the part you see before “Continue reading”), with special characters un-escaped so that it has proper HTML formatting.

This is what I need to change.

Issues

Posts With Images

Issue number one: images won’t render. This is because I use Lazysizes to lazy-load images, so that the bandwidth (and decoding time) is only spent if you actually scroll down to the point where it’s required.

This is implemented by changing all <img> tags from <img src="imagefile"> to <img class="lazyload" data-src="imagefile">, essentially making them nonexistent until the load script “fixes” them to make them load.

Posts With Featured Images

When I first started this, I didn’t realize that RSS had a method for attaching images to an item. So my question was… how do I do it? I had tried injecting an <img src="featuredImage.png alt="Featured image"> tag into the <description> markup, but.. it didn’t work (for other reasons), and I did some digging.

The correct way to do this is to use <enclosure>, specifying the url, length (file size), and type, in the RSS <item> markup itself.

Posts With Image Galleries

Okay fine, I can get single featured images to work, but galleries with multiple images? Nope. I can’t find a good way to do that, want to guess my workaround?

I… gave up. Anything with a gallery just slots a note into the beginning saying that you have to go to the site because it can’t (easily) be done, especially without wasting a lot of space.

RSS Feed Size

There’s about 70-ish posts in the feed right now, and with their full text suddenly visible, that makes it… huge. So huge, in fact, that just about every mobile app I tried (desktop was fine) refused to actually load it.

Solution? Just include the most recent 10, which at this current rate is a week and a half of content. If your reader doesn’t auto-delete, then it will keep the ones that dropped off, but doesn’t have to download them all each time it wants to check for new ones.

After

So here’s the final version of that template:

<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>{{ if eq  .Title  .Site.Title }}{{ .Site.Title }}{{ else }}{{ with .Title }}{{.}} on {{ end }}{{ .Site.Title }}{{ end }}</title>
    <link>{{ .Permalink }}</link>
    <description>Recent content {{ if ne  .Title  .Site.Title }}{{ with .Title }}in {{.}} {{ end }}{{ end }}on {{ .Site.Title }}</description>
    <generator>Hugo -- gohugo.io</generator>{{ with .Site.LanguageCode }}
    <language>{{.}}</language>{{end}}{{ with .Site.Author.email }}
    <managingEditor>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</managingEditor>{{end}}{{ with .Site.Author.email }}
    <webMaster>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</webMaster>{{end}}{{ with .Site.Copyright }}
    <copyright>{{.}}</copyright>{{end}}{{ if not .Date.IsZero }}
    <lastBuildDate>{{ .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" | safeHTML }}</lastBuildDate>{{ end }}
    {{ with .OutputFormats.Get "RSS" }}
	{{ printf "<atom:link href=%q rel=\"self\" type=%q />" .Permalink .MediaType | safeHTML }}
    {{ end }}
    {{ range first 10 $pages }}
    <item>
      <title>{{ .Title }}</title>
      <link>{{ .Permalink }}</link>
      <pubDate>{{ .Date.Format "Mon, 02 Jan 2006 15:04:05 -0700" | safeHTML }}</pubDate>
      {{ with .Site.Author.email }}<author>{{.}}{{ with $.Site.Author.name }} ({{.}}){{end}}</author>{{end}}
      <guid isPermaLink="false">{{ sha256 .Content }}</guid>
      {{ if (.Resources.GetMatch "featuredImage.*") }}
      {{ $resLink := path.Join "/content" .RelPermalink "featuredImage.png" }}
      {{ $stat := os.Stat $resLink }}
      <enclosure url="{{ .Permalink }}featuredImage.png" length="{{ $stat.Size }}" type="image/png" />
      {{ end }}
      <description>
        {{ if eq .Params.imageSlider true }}
        <p><i>Note: The gallery images for this post cannot be displayed (easily) on an RSS feed. Please visit the real post URL to view them.</i></p>
        {{ end }}
        {{ replace .Content "class=\"lazyload\" data-src" "src" | html }}
      </description>
    </item>
    {{ end }}
  </channel>
</rss>

So what changed?

Unrelated, but the <guid> tag now has a false isPermaLink attribute and has changed from the post URL to the SHA-256 hash of the post’s contents. This means that, in theory, if I push an update, your reader will notice and flag it appropriately.

On line 28, we limit to iterating through only the first 10 pages. Because they’re ordered most-recent first, this truncates the list to the most recent 10 posts.

In line 35, we check if there’s a featuredImage, and if so, grab the file size and link it in an <enclosure> tag, on line 38.

Lines 41-43 insert the note for galleries if one is present, and line 44 is the actual content.

Just to have the full text, I only need {{ .Content | html }}. The whole replace magic at the front is to “fix” the images so they load properly.

And with that, we now have… full text RSS feed!

If you’re already using it, you will need to delete and re-add the feed for these changes to reflect on existing posts, new ones will look like this going forward regardless.

Tek's Domain