Web Scraping

I am not super familiar with PHP, but if you need to do DOM parsing, you are better of using a library. Avoid rolling your own regular expressions like the plague, it will only cause you trouble in the future.

Looking around I found this PHP DOM manipulation library github.com/ivopetkov/html5-dom-document-php/, have never used it, but it looks promising and has a bunch of stars in github. And it seems to have documentation, which is always a plus.

Even if you are using a library, the problem with scraping is that it depends on the layout you are trying to get data from, so if that changes, you need to update your code. Because of that reason, in the past I found that it is better to keep your queries just specific enough to get what you want, in the case the layout changes a little bit, but not too much, you might get away without changing anything on your side.

What does this has to do with C3?

The app I'm using was created in c3, which can do anything really. I parse everything via c3...just easier to manipulate inside. I have considered using php to get what I need but that's kinda out of my pay grade.

I'm using the meta tags, and not raw data...which should be stable (as in not many changes since that would break all websites using the info.)

I've done some research on that, and frankly I'm lost as to how to implement DOM parse. I see "simple DOM" for PHP but seriously above my paygrade lol.

I managed to get the following down from websites that have a widget added.

Which works 98% but... occasionally has issues loading the image (I put a default image in place since some pages don't have an associated image or it fails to load).

If you wouldn't mind expounding on the DOM, I'd appreciate the insight.