Templates Plugins Courses

Get Web Page's Meta blocked as a bot by website

When I try to pull some site’s data, like bloomberg.com, it recognizes it as a crawler bot and prevents the real page’s data from showing. For example:

https://www.bloomberg.com/opinion/articles/2021-02-09/clubhouse-success-shows-niceness-can-exist-on-social-media

Bloomberg - Are you a robot?

We’ve detected unusual activity from your computer network

To continue, please click the box below to let us know you’re not a robot.

But if you post this url to twitter or slack, it is able to pull the image and title. So i wonder if there is a way to fix this or prevent this issue?

Thanks

Hi @jaym,

Thanks for reaching out. Please let us investigate this request with our developers as well, then get back to you with another feedback asap!

Best,
Alex

Hi @jaym,

An update here. Unfortunately, it is not possible to influence on this matter at plugin level. We tried to use custom code on page - doesn’t work. There are some websites that use this type of security at entry, which is beyond our control I’m afraid.

Thanks for your patience. :pray: