QBNers for QBN

  • Started
  • Last post
  • 14 Responses
  • skinny_puppy30

    Qooble is a QBN search engine (in beta). https://qooble.haasie.com/

    It does not yet get fresh data, so you get what you get.

    However, thanks to @palimpsest, it already contains QBN history covering a span of over 20 years: all the way back from 9 December 2002 right up until 9 October 2023. Over 5 million replies and notes.

    It also supports some filters to help narrow down your search: https://qooble.haasie.com/about#…

    Hope you find this tool useful. Keen to hear questions, thoughts, suggestions.

  • palimpsest5

    As the end of the year approaches I wanted to share an idea for a collective project. I would like to make an advent calendar out of your contributions. It doesn't necessarily have to be advent themed, just something you would like to share with the rest of us. It can be something new or something you have already posted and want to showcase. All mediums are accepted.

    I plan on setting the deadline around November 20th to give you enough time to prepare something and give me time to put it all together. I will post a signup form if there's enough interest and assign days to participants.

    • I like this idea.skinny_puppy
    • How many units would be needed? Digital only?maikel
    • I think a minimum of 12 for it to work, we would skip a day. It can be several people per day & we could extend it all month (secular version) if there's more.palimpsest
    • I'm open to all formats: image, audio, text. But it would have to be digital so everyone can "open" the work the day it's available.palimpsest
    • Cool ideaKrassy
  • palimpsest5

    After a conversation about remixing mugs through AI or PS I am looking for a method with which we can agree on to opt out if we don't want our mugs or our work remixed. The default behavior is the community having a ball with anything we post and I think all of us are on board with that. But there might be some cases where we don't want our stuff to be messed with. My proposal is very simple. Just add a #noremix tag to your posts and hopefully the community will respect that choice. Everything goes will be the default behavior if you don't care to opt out. We can fine tune it with #noAI, #AIOK, #noPS or #PSOK if you only want to opt out of certain things but allow others. 
    Discuss

    • drgs has brought up a good point.
      #noRemix should include no reposting.
      AKA leave it the fuck alone.
      palimpsest
    • telling people what to do on the internet always turns out so well. the only way to protect what is posted is to not post it.imbecile
    • That's a sure way for it not get messed with. I'm just trying to see how far we can take this and how willing we are to cooperate. Elastic boundaries.palimpsest
    • The intention behind this never was "telling people what to do on the internet".palimpsest
    • I clearly knew what I was setting myself up for and I'm not complaining about it.palimpsest
    • morally, i agree with you. realistically, i know users like hotroddy, Hayoth, and crazyprick existimbecile
    • I personally don't mind. I was just trying to get the conversation going in case it intereseted other people and would benefit from more participation.palimpsest
    • I'm doing this for the people.
      You can print my face and wipe your ass on it for all I care.
      palimpsest
    • Are we still allowed to photoshop your pictures for personal use?drgs
    • If we are talking about the spirt of the proposal, no.
      If we are talking about me personally, fap away!
      palimpsest
    • @imbecile
      Whatever your beef may be with hotroddy, Hayoth, and crazyprick they're not the ones doing the remixing.
      palimpsest
    • AND the remixing is mostly done in good faith and accepted by the majority. I only wanted others to have the option to not be remixed or reposted.palimpsest
    • Sorry if that wasn't clear in my original post, sparky.palimpsest
    • I think it’s better to make requests in prose instead of using tags. It’s more clear that you’re asking for a favor and that there’s no policy about it.pusherbot
    • QBN online consent formdrgs
    • Interpretive dance saved as a gif.palimpsest
  • skinny_puppy8

    Been catching up on other work, but here is a quick update.

    Haasie is now closed for uploading Imgur links, but you can upload images and videos directly. Nothing bigger than 200MB atm. Copy n Paste works too:

    I'm still devising a plan to move all the images I managed to download from palimpsest's links over to the live site. Download took a few days due to loadshedding.

    • you da man!YakuZoku
    • Thanks @YakuZoku!skinny_puppy
    • i can scrape all imgur links on the site and provide referring urls if that helps? might take a long time tho as the site is biiigtrooperbill
    • ^ That's what palimpsest has been doing.Continuity
  • skinny_puppy1

    @palimpsest could you send me a dump of all the links? I've been downloading them 1 user at a time but it is slow going. That way I can just package them all into 1 upload and let it chug away in the background on my local.

  • skinny_puppy0

    Wow. I have tried creating a number of new servers. Thought I could proxy through them to continue the downloads. But they are all blocked. Both on DigitalOcean and Linode.

    Even my trusty old server with a few blogs on it that has never even seen an Imgur link is blocked. It's like all these IP addresses have just been blanket blocked.

    • Interestingly, ever since I plugged in my images links into Haasie on Thursday (or Friday), not a single imgur image shows up for me when I'm on my home WiFi.Continuity
    • It's all broken.Continuity
    • I've seen similar comments online. I guess Imgur is just blocking swathes of IP addresses.skinny_puppy
    • Interestingly, everything's working again since the last couple of hours. I don't know WTF imgur are doing.Continuity
    • My servers are not that blessed. But my local is working. I'll download what I can here and see how to get it into Haasie afterwards.skinny_puppy
  • skinny_puppy3

    Quick update. There are 2 parts to Haasie at the moment: 1. Ingesting links. 2. Downloading the files for the links from Imgur and storing them.

    Good news: I've improved the link ingestion. Haasie breezed through 58K links in a test.

    Bad news: Haasie has been blocked from Imgur from downloading more files. So all the files downloaded recently have been 0 bytes.

    I've taken Haasie down for the moment, to give Imgur a breather. Hopefully in an hour or two I can start it up again.

    The files that were uploaded at 0 bytes are easy to identify and I can reprocess them and download them properly when Imgur lets me again.

    Will aim to build a more elegant solution for throttling soon.

    • I blame PhanLo.
      <3
      palimpsest
    • Haha!skinny_puppy
    • sorry :-(PhanLo
    • Are you storing all these images on your servers?drgs
    • @drgs to get technical they are eventually stored on an S3 equivalent called Spaces (from DigitalOcean). And then cached by a CDN called Fastly.skinny_puppy
    • To be less technical, effectively stored on servers I am running. Setup specifically for this purpose.skinny_puppy
    • Couldn't you upload them back to imgur, but this time under your account, and your server only translates old codes to the new onesdrgs
    • Imgur pics launderingdrgs
    • Interesting idea, that could definitely be done as a last solution. sted suggested something similar. The first challenge remains to backup the files.skinny_puppy
    • TBH though, I am having a lot of fun atm. I guess if stops being fun, I can look for a longer term archive solution - I like your suggestion for that.skinny_puppy
    • lol @ launderingskinny_puppy
  • skinny_puppy13

    Stress tested with PhanLo 2022. Uploaded 3182 links in one go. They were processed in about 5 min. This was the last image of 2022 (on the list):

  • Continuity2

    2021 was light year for me. On 19 images sent to imgur!

    • only*Continuity
    • That is light.
      Heads up: 2020 only has data starting from 2020-11-25.
      I'll scrape the whole year today or tomorrow.
      palimpsest
  • skinny_puppy2

    @palimpsest two changes if you can:

    1. Remove the href=" prefix.
    2. Don't trim .jpeg to .jpe

    I can add support for both of those (or people can just fix their own exports), but I'll only get to it tomorrow night.

    • Yes, sir!
      I'll take care of it today.
      palimpsest
    • Anyone know why imgur made the change from .jpg to jpeg extensions a while back? They still all resolve, but still...Nairn
    • I only know, it 'ignores' the extension. you can change url to .png .gif .jpg .jpeg and it will load fine. only on download you get the real format.uan
    • Thanks!skinny_puppy
    • Removed the href and fixed the jpeg issue. At first I only had jpg on my regex then thought I should add png and gif so changed it to w{3}palimpsest
    • not counting on the pesky .jpeg
      I also did a quick fix to only preview the last 100 posts (or the PhanLo safeguard).
      palimpsest
    • PhanLo 2022:
      2429 posts with imgur images and a total of
      3271 imgur images.
      palimpsest
    • YAASSSS.PhanLo
    • Hahaha fucking hell, Phan!Continuity
    • Awesome! This is how we stress test Haasie.skinny_puppy
    • Thanks @palimpsest much appreciatedskinny_puppy
    • 2020 has been scraped and updated.
      https://i.haasie.com…
      palimpsest
  • skinny_puppy1

    This so is awesome! Thank you!

    The biggest upload I've seen to haasie,com was around 150,000 links. Or they just uploaded repeatedly very quickly.

    But I am reasonably confident that around 2000 links a pop should be fine. Unfortunately I have not had a large enough dataset to test with.

    Planning some improvements in that domain soon which will allow for more.

    • 2000, impressive!
      Don't search for PhanLo, it'll break your computer. I need to add pagination for the preview.
      palimpsest
    • Cool, if it doesn't break my computer might be a good test dataset.skinny_puppy
  • grafician-4

    Nice work!

    But, how do we switch the links in context in all QBN posts?

    Also, an alternative to Imgur exists?

  • uan8

    thanks palimpsest! Nice Work!

  • sted0

    TypeError: dataset is undefined
    RuntimeError: dataset is undefined

    • You have to define them. Choose a dataset and type a username.palimpsest
    • https://i.imgur.com/…palimpsest
    • I did, and it wasn't working... but it does now. thankssted
    • wow i had no idea that observablehq shows the code while you're writing it :Dsted
    • Observablehq is a gift from god.palimpsest
    • the God that also gave us d3js:
      Mike Bostock
      also showed the world what data visualisation can be working for the nyt.
      uan
    • Amen.palimpsest
    • Definitely going to play with observablehq now too.skinny_puppy
  • palimpsest

    Word is bond.
    Panopticon imgur search: https://bit.ly/3VNoKgq

    This tool retrieves the user's imgur posts to compliment skinny_puppy's awesome haasie tool.
    https://www.qbn.com/reply/409086…
    https://haasie.com
    Search for a user, select a year and get the post. I don't know what the entry limit is for haasie.com so go easy.

    I created a new thread  in the hope that other users will post projects that compliment the QBNverse.

    For the people.