Mismanagement of AI resources

From: aidotengineer

Mismanagement of AI resources can lead to “company crippling, career ending failure” [00:00:43] and even “total collapse” [00:10:20]. This often involves embracing “worse practices” to “completely torpedo your projects” [00:00:57] and “alienate everyone that you work with” [00:01:03].

Seeding Division and Disconnect

A crucial step in mismanagement is to “divide and conquer your own company” [00:02:27]. This means:

• Embracing Disconnect Disregarding the relationship between willingness to pay, price, and cost, by contemplating “unreasonable goals” [00:02:35]. This involves adhering to an “anti-value stick” [00:03:18] where:
  • WTP (Wishful Thinking Promises) Telling customers that AI will do “absolutely everything for them” without worrying about details [00:03:32].
  • Price (Particularly Ridiculous Infrastructure Costs) Buying the most expensive GPUs without cost-benefit analysis, “max[ing] out the company credit card” [00:03:54].
  • Cost (Cascade of Spectacular Technical Debt) Building systems that are “so convoluted, so intertwined that even you as an executive can barely understand it,” ensuring job security when it inevitably breaks [00:04:13].
  • WTS (Why This System?) Justifying AI projects with “because AI,” as “no board is ever going to question you” [00:04:45].
• Creating Silos Attending every AI industry conference but “never go back and talk about what you learned with your team” [00:02:48]. The objective is to “create impenetrable silos and incentivize secrecy between your teams” [00:03:01].

Defining a Flawed Strategy

When defining an AI strategy, it’s essential to:

• Fake Diagnosis Grab old reports, highlight random paragraphs (especially those least understood), and declare them “a must fix” [00:05:09]. Crucially, “don’t bother talking to anyone who actually does the work” [00:05:20].
• Ambiguous Guiding Policy Use “incredibly ambiguous and vague” statements like “become the global AI leader in everything” without defining “everything” [00:05:27].
• Unrealistic Action Plan Announce plans for AI-powered SEO tools, generative art plug-ins, and AI drone lunch delivery services at company meetings, using words like “disruptive” [00:05:41].
• Avoid Timelines “Embrace Perpetual beta” and create a “massive backlog in GitHub” [00:06:13], as “timelines are for companies that intend to finish projects” [00:06:07]. Creating 4,000-page documents and posting them everywhere can “erode people’s willpower to engage” [00:06:25].

Communicating in Jargon

Effective mismanagement involves communicating in a way that “nobody understands” [00:06:48].

• Tsunami of Jargon Drown everyone in complex terms like “multimodal agentic Transformer based system leverages F shot learning and Chain of Thought reasoning to optimize the synergistic potential of our Dynamic hyper hyper parameter space” [00:06:53]. The goal is to “look incredibly smart even if nobody understands a word you’re saying” [00:07:08].
• Strategic Obfuscation Use jargon to “hide the jobs to be done” [00:07:41]. For example, instead of saying “write a prompt,” say “we’re building agents,” which ensures “mental health experts were not in the room and didn’t know how to participate” [00:07:47]. Similarly, instead of “make sure the AI has the right context,” talk about “Rags,” and instead of “make sure users can trick the AI into doing something bad,” say “prompt injections” [00:08:12].
• Engineer-Centric Prompting Encourage engineers, “not the people who might best understand your customers,” to write prompts [00:08:24]. The aim is to make “everything even writing prompts seems super technical and Out Of Reach for everyone” [00:08:53].

Flawed Mobilization and Deployment

When mobilizing for AI projects, adopt a “zoning to lose” [00:09:20] framework:

• Random Task Assignment “Randomly assign AI tasks to people with absolutely no relevant experience” [00:09:27], such as outsourcing data review to offshore Q&A teams with “very little context about your business” [00:09:32].
• Direct-to-Customer Launch Launch “completely untested bug written AI chat Bots directly to your customers” [00:09:48] from the incubation zone, “never worry about beta testing,” and “disregard quality assurance” [00:09:53]. The worst outcome is a “potentially career-ending PR disaster” [00:10:01].
• Resource Misallocation “Yank all your best Engineers from potentially supporting your Revenue producing products” for AI projects, leading to “total collapse” [00:10:12].

Focus on Tools Over Processes

To ensure failure, “focus on tools not processes” [00:10:39] and “burn it all to the ground” [00:10:35]:

• Tool-Centric Problem Solving Instead of analyzing problems, “just throw tools at them” [00:10:49]. If a RAG system fails, “just buy a new more expensive Vector database” [00:10:51]. If agents aren’t working, “just pick a new framework and vendor” and “F tune without any measurement or evaluation” [00:11:15].
• Blindly Trusting Metrics Use “every off-the-shelf evaluation metric you can possibly find,” never customizing them for business needs, and “blindly trust the numbers even if they make no sense” [00:11:01]. Evlas are a “vendor problem” [00:12:09], with a “one size fits all solution” [00:12:12]. Create dashboards with “every off-the-shelf metric,” ensuring they are unintelligible and don’t track outcomes [00:12:26]. Hoard metrics until one “is going up and to the right,” then “claim success” [00:12:47]. Adopt all evaluation frameworks, letting them guide blindly, and never ask if they measure success [00:13:01]. Optimize for metrics like “cosine similarity, BL, and Rouge, ignoring actual user experience” [00:13:17], and “never cross check with domain experts or your users” [00:13:24].

Avoiding Data

The “most potent technique” for mismanagement is to “avoid looking at data” [00:13:39].

• Blind Trust in AI Let a tool handle data because you can “absolutely 100% trust the ai’s output without ever looking at it yourself” [00:13:56]. Looking at data is an “engineering problem,” not for leaders [00:14:03].
• Trust Your Gut “Trust your gut” as it is a “reliable substitute for data, especially when you’re making million-dollar decisions” [00:14:34].
• Exclude Non-Engineers from Data Ensure “no one else is looking at data” by putting it in “complex systems that only Engineers can access and it’s not available to domain experts” [00:15:20]. Insist on buying custom data analysis platforms requiring “a team of phds to operate and understand” [00:15:39], especially if it takes six months to load and has “errors incessant” [00:15:51].

Following these practices “meticulously” is “guaranteed that you’re going to waste time, resources and alienate all the people you work with” [00:16:04].