After incredibly frustrating errors with ChatGPT, I switched to try Anthropic's Claude for help drafting and refining documents. Just like ChatGPT, you do not have access to "Projects" -- which collects threads and files into a common database for reference -- without a subscription.
And of course I have screenshots (see bottom of slides).
However, despite fewer sudden gross procedural errors than ChatGPT, a Pro subscription to Claude might be worse:
- (slides 1-3) It frequently retrieves data incorrectly from the Project files.
- Despite being way under file limits (around 12% currently) and telling it exactly which files to look at and for what, it can still collect data that is factually incorrect.
- Then proceeds to use it in the document draft in Canvas.
- Now I have to luckily spot these gross factual errors and
- Explain it to Claude.
- Get it to re-read data.
- And then fix the draft.
- (slides 4-6) Even when it does this, there might still be a version mismatch error that also requires correction.
- What frequently happens is Claude claims it has made certain changes.
- You even see it editing the draft document in the canvas.
- But all changes are reverted in canvas draft -- the version number may have changed, but nothing has actually been edited. If you don't realize this, you could continue to work on an outdated draft.
- Meanwhile Claude thinks it has successfully applied edits because it got confirmation it did, AND it has a different "internal" view of the draft.
- You have to realize it has this internal and different version, then get it to dump it into a new canvas version to see what it sees and see the changes it thinks it applied.
- (slide 7) Aside from the above errors, even when you give a specific instruction, Claude can make a comprehension error.
- You are NOT able to directly edit the draft in Canvas (unlike Gemini or ChatGPT) so you either have to:
- Describe what Claude did wrong and how to fix it.
- Fix it offline and upload it to Project Files and tell Claude to replace the canvas version with the file. And just that last step can take a long time depending on the size of the draft.
- Or if you don't have access to Projects, cut and paste into chat, which can eat up crazy amounts of context memory, resulting in having to start a new thread completely.
- All workarounds are extremely tedious compared to:
- If Claude hadn't screwed up in the first place.
- If you could just directly edit the Canvas.
- (slide 8-9) Even a simple (but important) instruction to load a document into the canvas so you can work on it does not work properly.
- I asked Claude to load a text file into canvas verbatim. It confirmed it did so.
- I asked it to double check and it identified a bunch of discrepancies.
- Now I can't be sure if there are discrepancies it failed to find.
- Am I supposed to waste my time telling it to fix its failures?
- Chasing and fixing all these errors costs you prompts, and even on a Pro subscription, you have sharply LIMITED usage. So literally you are:
- Paying to test a bugged product.
- Paying to submit bug reports.
- Wasting your limited prompts fixing errors.
- Getting locked out of doing actual work when you hit the limit, which resets literally HOURS later.
- Even without extensive error fixing and bug reporting, the 5-hours limits on the Pro subscription are ridiculous.
- For example, I was told my limit resets at 11pm. I log on around 11:07pm. At 11:55pm it again tells me I've reached my "5-hour limit" and it resets at 4 am.
- I've barely just started working!
- Too bad ChatGPT had gone bonkers otherwise I'd still be using it because it was more or less unlimited once you subscribed.
What the heck are you really paying for?
They should be paying YOU.
Is the subscription actually a scam?
Comments
Post a Comment