Is it ethical to make my own feed?

You want to follow something in LightWatch that doesn’t have an RSS feed. To do that, you need what’s called a “web scraper”. Is scraping ethical? People have very strong opinions about this, but like everything, the answer is, “It depends.”

My short answer: if your scraper effectively emulates your own personal browsing, and you keep your feeds to yourself, it’s fine. This is a personal stance, not a legal one.

To understand this reasoning, let’s look at the common objections to scraping from the viewpoint of the content source.

“We don’t want people to redistribute our content”

Easy! If you keep the scraped content private, you’re not redistributing it.

“We don’t want anyone to train AI with our content”

Also easy! You’re not training AI.

“We want people to see our advertisements”

OK, this one is actually hard, and you’re going to have to decide how you feel about it. An ad-supported site only survives if people see its ads. If you scrape content to view in LightWatch, those ads don’t get seen, and the site doesn’t get those pennies.

My feeling is that as long as you keep the scraped feed to yourself and aren’t offering an ad-free alternative to others, it’s no different from running an ad blocker while browsing the site. Whether that’s a problem is up to you.

“We want people to stay in our walled garden”

This is dumb, and I don’t think there’s any ethical dilemma here. Whether you see a JPEG on a website or in an app is a technical detail.

“We want people to pay for our content”

This is totally reasonable, and it’s why my stance is limited to personal use. It is not OK to “liberate” paywalled content for others.

But what about content you pay for, for personal use? So long as you keep it inaccessible to others, my opinion is the same as above — a JPEG on a website or in an app is a technical detail.

“We want to protect our servers from abuse”

This is the most important one. Scrapers are a huge problem — not because they’re conceptually bad, but because swarms of naive scrapers create huge headaches for the servers they hit. This has always been true, but is particularly problematic in the age of AI, where anyone can make a scraper, but few people know how to do it respectfully.

It’s important to be a good citizen of the web, and that’s a big part of what the build your own feed generator guide is about.

The bottom line

I’m sure there’s plenty to quibble with here, but I still think it boils down to a JPEG in a browser versus a JPEG in an app being the same thing. Be respectful with your tools, don’t share things you don’t have permission to share, and you can sleep easy.