fetch page as markdown is taking forever to load
-
#LLJW
<span style="color:white;background-image:linear-gradient(to right, lightblue, navy)">My name is Jeff</span> -
Yes, we are trying to optimize it.
-
@info-creaticode I’m not sure but maybe if the markdown part is the issue, then you guys could just omit that feature and then the block would just return the code of a website like all the HTML and all, just raw. Most users might not understand the output or how to use it, however, in a tutorial for teaching how to use the block, you could show them that you can utilize ChatGPT to analyse the website code and then retrieve what you need from it. You could also teach that using ChatGPT for simple tasks like retrieving the title of the page isn’t neccessary and is more prone to error and taking longer when you could just use a regex search to find the page title.
#LLJW
<span style="color:white;background-image:linear-gradient(to right, lightblue, navy)">My name is Jeff</span> -
We are still working on fixing the issue. It is more about fetching complete data, since many websites using JS to present dynamic content, so it is not as simple as retrieving a static HTML file.
-
@info-creaticode Ohhh that makes sense, websites do change their content after the page is fully loaded. However it does negatively affect using that block for things that are complete but are static, like when I use it to fetch from a fun fact API or just API’s in general.
I also don’t know how easy or possible that would be to solve because to display the dynamic content you’d also need to run the JavaScript which can lead to problems like JS scripts of that page conflicting with those of the playground such as two variables with the same name.
I think perhaps a new block could be made instead for fetching complete data since at the moment, it messed up my browser project a bit that was relying on that block since although I used iframes, I would use the result of the fetch block and then search it to get vital information such as page title and page language and then I would check if the page is safe (not a virus or spam) using the Google transparency check.#LLJW
<span style="color:white;background-image:linear-gradient(to right, lightblue, navy)">My name is Jeff</span> -
Please wait a bit longer and test it after we release a new version of that block. If it still doesn’t work for your use case, we can discuss adding a different block.
