• T156@lemmy.world
    link
    fedilink
    English
    arrow-up
    11
    ·
    2 days ago

    The cost has also shot up because a lot of the new frameworks are much more token heavy than the old ones.

    So the original free plan might have made sense when people were only typing little questions into it, and using a handful of tokens, but is no longer cost-effective with things like modern agent pipelines constantly throwing tens of thousands of tokens at the service.

    I tried running a little locally hosted agent thing on my computer the other day, and it was feeding a hundred thousand tokens at the model every few minutes, because it was keeping all the files in context. Sure, it hit the cache a lot, and so the effective cost would be less, but it’s still a lot more token usage than me poking the model with inane questions.

    • Mikina@programming.dev
      link
      fedilink
      arrow-up
      4
      ·
      15 hours ago

      I’ve been kind of forced to use AI for my work, and since I had an unlimited access to whatever model I needed, I figured “why not try”. I was able to find my workflow that works, use it to explain the architecture of the code surrounding my bugfix/feature (I work in gamedev, on a project I’m only a contractor for, so I don’t know the codebase too well), make me a documentation, then draft a plan how to implement it. Implement it, then I just look through it for ideas, combine it with my domain knowlege to see if it missed any obvious things or solutions (which it usually does in a larger projects), and then build my own solution from scratch based on what I know, and what it suggested.

      The part where it explains the architecture and data/systems flow is invaluable and it does make ot faster than I could have parsed through unknown code, while being verifiable enough to be trustworthy. It’s a good kickstarting process. Do I need it? Not really, but it would take me longer.

      But. It eats tokens like hell. My average monthly token usage is around 800m tokens.

      I’ve been told I’m not using the AI enough in my workflow, because I write my PR comments and don’t use AI for code reviews.

      I’m seriously considering just leaving IT altogether. It’s just eroding my reverse engineering / codebase orientation skill, while I’m replacing it with something that costs the same amount as my sallary for doing it slightly faster/more easier, and the price will only get a lot worse. I don’t get it. How can’t they see it?? How can they look at “Oh, he’s costing us 2000$ a month in AI use [at current prices], but solves 3 instead of 2 tasks a week, while slowly loosing vital skills”, and say “that’s worth it! But he could use it more.”

      I hate it. Fuck managment.