The loads are slowing down the generators that are burning a well metered amount of fuel to stay at 60Hz. This is a delicate balance since the phase angle must also be spot on.
If a generator and the local line disagree on f, phase or V, you have a short circuit.
If you lose a large amount of load, your generator will spin up with the excess fuel until the control system re-establishes the right amount of fuel.
But now your generators are out of sync! No worry, for small disturbances the dissipative losses sync everything up like syncros on a manual transmission.
But the disturance cant be too big!
Rotating machines are big and heavy, so the first line of defense is their inertia. But this is a finite (and precious) resource.
Contrary to belief, renewables, or generally speaking DC, makes things this stability problem worse. They generate large amounts of power while providing no inertia.
You'd think it isn't a big deal since the DC-AC converter can just synthesize whatever is needed. Heck just keep it rigid at 60 Hz with no phase change.
Well the later doesn't work - the rest of the grid is no longer at that phase and frequency so you got yourself a short.
Furthermore, the DC-AC converter, despite their manufacturers' promise, has no good way to establish what f and phase it should be at during a disturbance (and these magic codes are closed source, believe it or not)
Anywho, a large enough loss of load causes the grid to enters into unstable oscillations, causing protective relays to trip causing a zipper effect where the grid goes down.
Now restart will take a few days depending on the energy mix (fastest for hydro heavy)
Long story short - this is not a trivial problem, and the data-centers can't be allowed to just dump load willy nilly.
EDIT: made it clear that the grid killing disturbance is not caused by renewables; not exclusively anyway. Everyone has to play nice or the grid goes down.
Grid forming batteries and inverters are a thing. They can control the frequency on the grid. Just a matter of getting the right equipment.
The nice thing with data centers is that they are somewhat flexible. It's not a constant load. Data center operators can choose to reduce load. And if properly engineered, they could do so automatically based on signals from the grid.
The issue with outdated grids is that it relies on technology (spinning mass) that's at this point a century old. Which makes it brittle against outages like you describe. The solution is not more spinning mass but batteries and renewables to take the place of that spinning mass. A battery can respond to oscillations in milliseconds. If you then add flexible load that can spin up/down based on the amount of available power, you gain a lot of stability.
> The issue with outdated grids is that it relies on technology (spinning mass) that's at this point a century old. Which makes it brittle against outages like you describe. The solution is not more spinning mass but batteries and renewables to take the place of that spinning mass. A battery can respond to oscillations in milliseconds. If you then add flexible load that can spin up/down based on the amount of available power, you gain a lot of stability.
Why the spinning mass technology being "century old" (more like "millennia old" but anyways) is a problem somehow? The Newton's First Law didn't change much in the time that has passed.
Anyone who tried to "respond to oscillations in milliseconds" knows how hard that problem is because the force you apply is integrated twice before it takes effect. Try stabilizing a swing by pushing it's forward when it's behind the equilibrium point and pushing it backward when it's ahead of it. Now imagine a grid of swings connected by rubber bands and a distributed system of independent actors responding to oscillations. There are much more ways in which this system can diverge rather then converge.
Time may prove me wrong, but the arguments like "spinning mass is old therefore should be replaced" certainly won't.
> Newton's First Law didn't change much in the time that has passed.
No, but technology has moved on quite a bit. Heavy fly wheels are no longer the state of the art here.
A few tens of GW of battery capacity (i.e. a few dozen nuclear plants worth of capacity) that can switch on/off in milliseconds can do a lot for grid stability. That's part of the reason why grid operators are rolling out so much batteries. It's not necessarily about supplying energy for a very long time but about smoothing out peaks and dips in energy supply and demand and responding more or less in real time to that.
This stuff is basically being rolled out at industrial scale in a lot of places. Australia, China, etc. pretty much run increasingly on mostly renewables. This is no longer as speculative as it would have been ten years ago.
Yes, there are engineering challenges with rolling that stuff out in a lot of places. And even more policy and regulation challenges. Actually that is, by far, the #1 challenge in places like the US and Europe. Grid operators are simply structured and incentivized wrong to deal with this stuff efficiently. Texas is actually not doing too bad relative to e.g. California. But they clearly have some challenges still.
> A few tens of GW of battery capacity (i.e. a few dozen nuclear plants worth of capacity) that can switch on/off in milliseconds can do a lot for grid stability.
I think this is a misunderstanding of the problem.
Now, don't get me wrong, I believe flywheels are no longer involved but at one point they were for batteries and solar. Not to store energy, but rather to form the output voltage and to give the correct "inertia" of the waveform to maintain a correct phase with the grid. Prior, both batteries and solar were frequency followers. They'd look at the previous peaks and valleys to determine what their output voltage (or resistance) should be. If the input voltage fell too far, both solar and batteries would cut off to avoid damaging equipment on the grid. This is part of why the winter storm killed the texas grid (to my understanding) the voltage dipped too low which ultimately caused renewables to shut off completely to avoid damaging the grid. That all was somewhat of a cascading disaster.
Flywheels have been used as an inertia source to allow for both solar and batteries to act more like a hydro or fossil fuel generator. That's the grid forming technology. I believe (can someone verify?) that there are now all digital versions of this. But it's delicate software. Getting it wrong can do really bad things like destroying other generators or breaking expensive fuses.
Respectfully, I believe you have confused "new" with "state of the art". China is likely using batteries because their battery production is subsidized, Australia's trading partner is China so basically equivalent.
Perhaps a bad analogy, but it seems these battery systems attempt to stabilize by "pushing", whereas spinning mass can also work through "dragging" the phase. So eventually, you will have just a few rotating masses setting the freq and phase, with more and more systems tracking that and pushing, which seems like a recipe for chaotic equilibrium.
No, they are doing that because they have a ten year head start on the rest of the world. The US & EU not having done their homework is entirely on them. The US is choosing to pump a lot of funding into fossil fuels instead. You could say that China is simply spending smarter.
The rest is just economics. We might suck at making batteries. But we suck even more at making new gas/coal plants or fueling those cost effectively.
Anyway, this stuff is being deployed by the hundreds of gwh per year now. Much of it in China but some non trivial amounts in places like Texas and California as well. As a result, the grid is actually getting more stable, not less stable.
> Respectfully, I believe you have confused "new" with "state of the art". China is likely using batteries because their battery production is subsidized, Australia's trading partner is China so basically equivalent.
This is comically wrong. You're just making things up. Australia is at the forefront of this new tech due to necessity and some good luck. Scroll to the bottom of this article and have a look at the graph. It shows Australia has 5 times the grid forming battery infrastructure deployed or under construction than China or the US, or pretty much any other country. It is very much SOTA, needed for a grid that is a rapidly changing mix of rooftop solar, hydro, coal, gas and wind spread over a country the size of the continental US. Rotating masses are not going to cut it.
Infrastructure is an investment with an ROI. Fuel is an expense that delivers energy once and that is never recovered. The only relevant debate here is on how to finance the relative attractive investment for renewables and over how long it delivers that ROI.
That varies per region but this is a sector that is now about 2x the level of investment for fossil fuel related infrastructure. A lot of which is increasingly looking like the ROI might never happen.
To add on to this overview, there are a new class of inverters in research called grid forming inverters which don't just follow the rest of the grid. One interesting technique is they can simulate the inertia of a traditional rotating machine/generator.
10+ MW voltage-source converters that can't do up to around 80% of their nominal capacity as mostly-reactive apparent power with stabilizing synthetic inertia scaled as desired/specified are a mostly software issue, stemming from lack of regulatory pressure incentivizing the engineering complexity of that.
Though if you want to do a smoothing action on real power flux you'll have to colocate battery capacity with the converter.
Which to be clear is fairly cheap to do as long as you get compensated for the substantial frequency stabilization capacity this represents. I'm talking like 15~120 minutes at converter nominal AC power of battery capacity.
The first 10~20% of reactive power are almost free from the converter electronics, btw....
That’s a simplified and somewhat outdated version, there’s a huge range of technology to mitigate each of these issues but infrastructure generally isn’t free. A turbine does provides inertia for free where you need to pay for a flywheel or battery system.
In the end a renewable heavy grid can be extremely resilient due to all the batteries, but smart battery systems need to be incentivize or mandated because ‘dumb’ batteries are cheaper.
The ELI5 version is you're arm-wrestling someone and they suddenly let their arm go limp, so your arm slams down on the table since you can't react that fast.
To build on this analogy, tug of war fits a bit better. Nothing dramatic happens if one person let go, but if half of one team just let loose at the same time without communication, bad things happen.
I think an even better example is what happened to the Tacoma Bridge. You have an oscillating frequency and phase that needs to be precisely tuned in order for everything to work well. If something is out of sync wrong, you can end up with higher peaks, lower valleys, or flatter locations which can ultimately cause catastrophic failure.
In the Tacoma example, the input frequency continued to add onto the bridges motion which ultimately caused it to destroy itself. In an electric grid, a misaligned phase can cause excessive spikes (imagine 480V when you expect 240V) or the generators to ultimately burn themselves out because 2 generators are fighting with each other, one trying to raise the voltage while another is trying to decrease the voltage. The really tricky thing is that load (particularly inductive or capacitive loads) look almost exactly like an unaligned generator.
Reminded me of this fun video of a guy spinning up a water turbine generator and getting it synchronized with to the grid the old-style needle phase meter, then connecting it successfully:
I interpreted from that EU grid post mortem that individual generators are coordinated using out of band comms channels that aren’t the power grid itself. Am I mistaken that they do this? Is it done there, but absent here?
Communication that detects the release and travels as close as possible to the speed of the natural signal in the rope, and is robust enough to recover without losing stability if the other team grabs the rope again while your "let go" signal is mid-flight.
One way to dampen this is to put a really strong guy on each side, with instructions to never let go of the rope. These are the flywheels of the grid.
As the article says, this is difficult and adds significantly to the cost. But it is very possible.
(It also adds significantly to the cost of spinning mass generators too. Delivering 250% of nominal current in a short circuit situation requires beefing up a lot of components).
Actually large data centers at least if done in a vaguely alirack style architecture, can do this with a decent fraction of their nominal power for very little hardware cost, as reactive power and real power add up via Pythagoras (`apparent=sqrt(real^2 + reactive^2)`) to the apparent power (rms voltage times rms current, which is what the 60Hz electronics and 60Hz transformers care about).
The first 10-ish % are nearly free.
And alirack style datacenters have large 3-phase converters between the grid and some 240 (nowadays often 350) V DC bus, with the battery banks directly (with just fuses and sometimes a little bit of balancing/nudging power (think 10% of battery power rating)) on the bus, and then the servers also directly consuming from that bus.
The large converters on the battery bus thus allow synthetically smoothing load transients to the grid using the batteries to smooth that power draw.
This has just minor additional wear on the batteries and a small power efficiency impact from hitting through the batteries, both of which are easily paid by anything market-rate of providing that grid service.
Because they already need the power electronics and batteries anyways, unlike a utility battery farm that at best can argue day/night load shifting of solar production as the reason for the electronics and batteries to exist.
In that same spirit it's also effective to put batteries on the DC bus (between MPPT and inverters) of large solar farms, because they need the electronics anyways and it's actually reducing the required inverter&transformer capacity of the solar farm by peak-shaving.
Would demanding that large spikey users of energy like data centers implement some sort of demand ramping/isolation from the grid in the form of a massive capacitor bank or flywheel generator between them and the grid help reduce the risk here?
The data centres aren't inherently spikey, in general use their consumption is reasonably predictable.
However, if a DC detects that the _grid_ is wobbly (voltage or frequency deviations) the DC will disconnect without warning, and switch to its batteries and generators.
The grid complains because it's suddenly lost hundreds of MW of load. For the DC to have isolation capability, it would need a load-sink which can consume roughly the same power as the DC in normal operation, and can take in that load at a moment's notice.
It's a hard problem to solve, and probably better managed at grid-level than DC-level.
> The grid complains because it's suddenly lost hundreds of MW of load. For the DC to have isolation capability, it would need a load-sink which can consume roughly the same power as the DC in normal operation, and can take in that load at a moment's notice.
Thats why always have all the EVs be connected all the time (except the 20 mins they drive). EVs can provide demand as a service and take excess power whenever its available (instead of solar curtailment), and also provide an immediate source of load when events like this happen. Its a shame US is anti EV, it has the best systems at scale that can be leveraged to transform the entire energy ecosystem.
My intuition is that there would be a fairly stable base load, but doing something like switching on a new training run of a frontier model would be incredibly spiky, thousands of GPUs going from somewhat idle to 100% in seconds.
This is where a smart grid would really shine, because the utility could drop the electricity price to 0 or even negative and turn on 100,000 air conditioners/heaters to effectively give people free climate control to absorb the load imbalance.
Some of us in Texas are all too familiar with the problem of balancing load with generation, the risk of a cascade failure causing a slow restart.
During winter storm Uri, they did a duty cycle where we only had power available for ~6-12 hours at a time on the days it was available. This was apparently to avoid that very problem.
So far as I know, the obvious mitigations like winterizing NG generation and/or peering with neighbor grids have not been performed.
I've often thought that the target* grid phase should be encoded into a high frequency signal, say at 50kHz. Generators without inertia can immediately switch to the required phase and inertial systems can work towards it or disconnect their outputs if they stray too far.
The problem in my mind is that, as it stands, the signal that everyone has to latch on to is itself affected by load and by different generators latching on with different time constants in a complicated feedback loop. Would having a single authoritative source be an improvement? Would that be a way to eliminate the need for inertia?
*I suppose the "target" phase would probably be based primarily on the output from the biggest inertial systems and take into account their ability to adjust phase and frequency.
Synthetic inertia has come a long way. An inverter knows just as well as a rotating generator what frequency and phase it should be generating, because it can mimic exactly the ideal equations of motion of a rotating generator with programmable inertia.
Short circuit is when two sources try to drive the same node in a circuit at different voltages or frequencies. In an idealized circuit (such as where the wires are assumed to have zero resistance), current flow from one source to the other is infinite. In a real circuit, current flow rises to a level where "something has to give." This could include heating the wires between the sources or tripping some kind of protection mechanism.
I'm not great with analogies, but imagine a train with two locomotives, and each one is set to run at a different speed. There will probably be a lot of screeching of wheels etc. To make multiple locomotives work on a single train requires engineering them to be synchronized with one another.
So apparently grid forming or synthetic inertia can be provided electronically nowadays, but presumably it would not be too terrible to put a rotating mass in between the (e.g.) solar power source and the rest of the grid? So the electricity from the solar panels runs a motor that turns a generator. I suppose power is lost through friction and other inefficiencies, and you'd need to co-locate the solar panels into a big enough farm.
Large projects will have a large load interconnection tariff that’s supposed to shoulder the costs of upgrading the infrastructure to support these new projects.
Data center discussions are weird right now because people assume these things don’t exist and propose them as solutions. They already exist.
The problem in the article is something different: The sites they’re talking about are designed to disconnect from the grid and use backup power when the voltage drops, which can be a problem because now there’s too much energy being supplied to the grid and not enough load to absorb it.
When voltage is dropping (or phase is falling out of alignment) don't you want to shed load to stabilize? The issue seems to be more that too much load could shed too quickly causing wild oscillations in grid conditions from an undervolt to an overvolt. Seems like the correct, and not terribly complicated, solution to this is to have a process whereby the grid operator can request large customers to shed load and move to backup power temporarily.
That seems like such a reasonable suggestion that I would be shocked if such a thing does not already exist and is simply just not reactive enough and too manual.
Interesting when you start to take this insight seriously. You could imagine data centers becoming important military targets in the future if the US ever balkanizes. Beyond that, securing compute could become a part of everyday trade agreements.
As things industrialize further, a state's net compute flow might become a topic of campaign promises. We may later find correlations between states' access to quality compute and the educational/financial outcome of younger generations.
To be sure, my leverage and inertia with a $200/mo ChatGPT Pro subscription is already a lot higher than it is without. We're not talking about just chatbots anymore, modern agentic models represent a quantifiable increase in individual agency. Whether or not the job market implodes, I am able to make progress on things even while asleep or busy with something else. This kind of agentic capital has never been available to the working class before at scale.
States like Texas, with their independent grid, might find themselves in a favorable position in such a balkanization event, but past events have proved that ERCOT is not ready to operate a statewide power grid. I was in Texas during the '21 cold snap and it was brutal. I was using my gas stove for heat. If resources for compute suddenly became constrained, and compute was deemed a matter of security and economic survival, it would be a cutthroat environment for operators of Texas' existing 400+ data centers.
Linear progression is unfair, just like for taxation. If you're going to make a law like this, it should be something like nlog(n) so that the big players that abuse the system pay more than the little guy.
We used to have something similar in France: solidarity tax on fortune (ISF), abolished by Macron at the height of the gilets jaunes movement in a big "fuck you" move.
Now it's much easier for wealthy elites to not pay taxes at all, as outlined in the past year with the Zucman tax debates (which elites opposed for daring to propose a 2% floor tax rate for those who don't pay 2%), and lately with the ministry of Finance Amélie de Montchalin lying to the members of parliament about the existence of a note produced by her own ministry showing that over 13000 millionaires pay effectively 0% income tax.
> It's outrageous that I'm paying an extra fee to export energy to a neighboring state to power a datacenter.
I don’t find anything outrageous whatsoever about building out transmission to strengthen regional interconnections. Lord knows we are decades behind the curve on this. If AI datacenters are what finally lets us maybe start to get a tiny bit ahead of the game I’m all for it. When the bubble pops there will finally be a bit of pressure off, and perhaps we can get back to a tiny bit of overbuilt capacity to cover for exceptional events again. And more reasonable projects maybe won’t be sitting in a decade long interconnection queue.
At some point we get to all collectively pay for our parents lack of investment in energy infrastructure. We have been living off our (great?) grandparents investments into the future and wide scale deindustrialization of the economy since I’ve been alive. At some point you run out of inertia.
Could always move to Texas if you hate the idea of your state ever possibly spending a dollar that benefits someone across the border. They islanded their grid pretty much precisely due to this attitude.
Watching the PJM and MISO interconnections over the past decade or so has been illuminating. It’s been a very slow moving disaster in the making that nearly no one is paying attention to.
As for your proposed solution I doubt anyone - including the datacenter operators - would argue since that is largely what tends to happen as it is. Just change utilization factor to capacity factor and sounds good to me!
As a retail electricity customer, I vastly prefer whatever is going on in their operational context.
I live 10 miles from the "border" between ERCOT and MISO and I've got a lot of experience with both, particularly in disaster scenarios. ERCOT is a total shitshow when it comes to the edges. The fuel mix in MISO is pretty terrible for the environment, but it is also unbelievably reliable. The Mississippi River is not to be underestimated in its logistical impacts. The price of natural gas drives the daily fuel mix proportions, but it doesn't affect the available base capacity and its financial status (fully amortized for over a generation). If natural gas gets expensive in ercot and the wind isn't blowing, there isn't anything else to give. MISO has 2-3x the coal and nuclear capacity that ERCOT does.
Your math on the house seems off by a factor of 4? But 30% utilization seems high, as that would be a $2k/mo electric bill at 20 cents/kWh.
Reframing what you're saying, that would be a connection fee that worked out to just under 12 cents per kWh used for the first year, which seems both somewhat reasonable but also probably not going to move the needle much on the large deployments?
Yes, but using the 30% utilization factor I get 200A * 240V * 30% = 14.4kW , meaning a $14400 connection fee rather than $3600 ? $3600 would be a 7.5% utilization factor.
And yes, I got the one time fee aspect. My point is that if you're basing this off of expected utilization (rather than say the size of the service regardless of how much energy is used), and that expected utilization is roughly correct, then that one time fee can be thought of as amortized cost per unit energy over a specified amount of time. And that cost isn't even that high if amortized over a year.
No, because that doesn't go far enough and still shifts some costs onto everyone else.
Make the DCs themselves pay to play nice to work with the grid: BESS FFR/M-FFR, for generation capacity, and for transmission upgrades.
Right now, they're using privatized-deregulated communism to privatize the profits and socialize the costs. Everyone else shouldn't pay for these goddamn DCs.
I'm a bit confused about why this is a serious engineering problem.
If a gigawatt class DC suddenly needs to take its sensitive IT loads off grid, it could be designed with load banks on site to stand in for the IT load. These would be exceptional use only, so the specific cooling technology (obviously we want to boil water) is not much of an ecological concern. A gigawatt will vaporize ~100 gallons of water per second. How long until the grid can adapt? Five minutes? That's not exactly a heroic amount of water for these projects.
> could be designed with load banks on site to stand in for the IT load
Yes, but that's not what's happening. These giant loads are suddenly going offline and are making it the transmission and generation networks' problem. The DCs aren't going to pay for it when they can socialize the costs and problems onto others.
The most obvious answer is these big loads need FFR and M-FFR BESS (battery) capacity near them or on-site to decouple and absorb these huge transient load swings into something the grid can handle. ERCOT should set additional rules that large load customers must use load smoothing rather than simply dumping load at auto-reclosers/auto-cutouts. Not doing so forces the entire grid to be at risk of low inertia transient over-generation.
Because the Texian grid-operator (ERCOT) is predominantly isolated from the Eastern & Western US grids [†], serving mostly-just Texans – this is to avoid US Federal regulations, by being an only-intrastate entity (technically, which is: the worst kind of "correct" [∑]). This is a socially-engineered "problem".
This means their needs for maintaining grid inertia (which frequency is a measure-er of) are more difficult, as Texas cannot just purchase any substantial/meaningful electricity – everything must be generated in-house.
Worse, the lack of Federal regulations means ERCOT/Texas can create their own rules/regulations/markets for the management of electrical infrastructure. Texas chose to make (IMHO) the most-purely crony-capitalistic market [for buy/sell of MWhours] in worldwide existance – which leads to decannual winterstorm Disasters (totaling hundreds of BBillion$ in damages / hundreds dead). The losses from wasted silicon-ingot depositions (alone) are staggering $,,,.00 – just another cost of doing business within Texas's jurisdiction [ß].
After 2021's storm, some "free market" customers were left with a week of billing costing more than the year's remainder!.
Abbott will just blame the windmills and not a cost-incentive fee-structure which does not incentivize cold-weather preventative maintainance. In fact, it does the opposite (since electricity can 25X price-increase). "Tell me the incentive and I'll tell you the outcome."
[∑] having grown up and been schooled primarily in Texas, I think the best review I ever agreed with was: "The [L]one Star State" – Texceptionalism will kill us all
[†] small DC-DC grid-ties (altogether a low, single-digit percentage of ERCOT's overall capacity) exist with both grids, as well as another smaller with Mexico
[ß] which is why most hospitals/datacenters have their own diesel/LNG generators, on-site – in Texas these have even less regulations because: "temporary" see (e.g.): Tesla's multiple Texas facilities' backupsystems – enormous point-source pollutors
----
If you want a good breakdown of the 2021 Texas storm's decimation of ERCOT/Texas/electricity, look for the great logs reddit /u/redditmudder did (in realtime) as this disaster was unfolding. I cannot remember exactly how low the frequency got... but I think we were closer to EU [50hz] than US [60hz] – as grid operators REAPED profits* from their failing infrastructure driving electricity orders-of-magnitude higher. This is all public information; it wasn't "windmills."
This almost seems like a straw man to me. Isn't the much larger problem the actual increased energy usage and making sure that all of this massive extra cost doesn't just get dumped on consumers?
I am a huge proponent of AI actually, but very suspicious that "financiers" are suddenly creating what amounts to an energy tax by finding legal ways to sneak extra fees or rates into our electricity bills to cover build out and commercial usage costs.
But as far as smoothing out demand, my (admittedly a layperson) theory is that we need to force them to adapt more solar and wind and at the same time more facilities for handling the variable production from that. Such as more large batteries and a shift to large scale long term storage of renewable fuel like hydrogen or other fuels produced directly from renewable sources.
If you have a large production and storage of renewable fuel, then maybe you can build that in such a way that it can handle significant input variations that could include excess grid power.
What matters is peak capacity. Using the grid as a free battery for when your intermittent sources of energy go away against your fairly constant loads only makes the grid weaker overall. The best part of these huge predictable load centers is utilization factor.
Storage of some sort would probably help some, but overall the best type of load for a grid is predictable constant usage. Bonus points if it can reasonably be part of a load shedding/demand response program.
Some kind of large aerostat / inflatabe chimney reaching up till the tropopause largely insulating for most of its length, but with heat engines on top and on bottom. I'm not proposing to use water / ice transition, but just consider a hypothetical strong suspension system, heaving a bucket of water weighing 1 mass unit can be powered by lowering a bucket of ice weighing 1 mass unit (ignore tiny density difference). So one can simultaneously bring 2 thermal baths close together, both at the bottom of the heat loop (where ambient temperature and tropopause temperature in the loop are close by each other) as well as at the top (where ambient cold tropopause temperature and surface temperature in the loop are close by each other). So both at the top and the bottom mechanical power can be generated with a heat engine.
The ground level vs tropopause temperature difference is not perfectly stable but largely maintained throughout day and night cycles, it is effectively base load, no nuclear required! We could generate energy while helping the planet cool.
Best to place these in seas, at least 1 structure length away from coastline, not in the middle of densely populated area. Salty sea water can be frozen (purifying it because ions get pushed into the brine) for desalination, brought to conventional thermal power plants, melted on their cold sides producing potable water while improving the thermodynamic efficiency of the thermal power plant (more electric energy per unit of fuel / rod spent).
Data centers are pretty useful for adding more renewables to the grid. They generally have both a UPS (for short-term outages) and backup generators (for long-term outages). UPS means they're already paying for inverters and some amount of batteries, and then in combination with a more renewable grid, it makes sense to put the grid storage batteries there, and they have the incentive to do it to take advantage of time of day metering by actually running on them during peak demand hours. Likewise, if you have a huge capacity backup generator and the grid is stressed because renewable generation is low (and therefore prices get high), you can turn it on. Which allows the grid to get more power from renewables the other 95% of the time.
Texas is the standing proof that "making sure that all of this massive extra cost doesn't just get dumped on consumers" is not a problem. Texas has the most load, the most marginal load added every year, and the cheapest retail power. Load grew 15% in the 5 years to 2024, and retail prices fell by 1¢.
I am sorry to be the one to inform you about "inflation". The data you provided shows that rates increased from 2019-2024 at rates slightly below general CPI inflation.
Let's assume we're talking about core inflation (since headline inflation included changes in energy costs and that would hide the signal). Core inflation was us 23% from '19-24, but the energy costs in Texas were up 37%. Actually, headline inflation was up 25%, which is still below that set of price increases.
Please support your assertion that Texas energy costs decreased in that time period.
According to EIA State Energy Data System, "Average end-use energy price estimates by energy source, annual", electricity in Texas rose from $25.51 per million BTU (yes, I know) in 2019 to $28.95 in 2024, the most recent year available in the database. That's only 13.5%.
Ahh, I was focusing on residential energy (the cost borne by most of the people in Texas), which is up 25% per that source. The bulk and stability of industrial power is keeping prices down in aggregate.
[{"series_id":"ELEC.PRICE.TX-RES.M","name":"ELEC.PRICE.TX-RES.M, monthly","units":"cents per kilowatt-hour","f":"M","description":"Electricity sales to ultimate customer by state and sector (number of customers, average price, revenue, and megawatthours of sales). \n Sources: Forms EIA-826, EIA-861, EIA-861M","copyright":"None","source":"EIA, U.S. Energy Information Administration","start":"202603","end":"200101","data":[["202603",16.39],["202602",15.41],["202601",15.69],["202512",15.87],["202511",16.04],["202510",16.1],["202509",15.83],["202508",15.46],["202507",15.36],["202506",15.26],["202505",15.53],["202504",15.52],["202503",15.28],["202502",14.88],["202501",14.67],["202412",15.36],["202411",15.68],["202410",15.67],["202409",15.06],["202408",14.88],["202407",14.7],["202406",14.55],["202405",14.83],["202404",15.13],["202403",15.02],["202402",14.48],["202401",14.35],["202312",14.64],["202311",14.79],["202310",14.89],["202309",14.68],["202308",14.17],["202307",13.95],["202306",14.35],["202305",14.65],["202304",14.36],["202303",14.61],["202302",14.72],["202301",14.36],["202212",14.49],["202211",15.01],["202210",14.94],["202209",14.75],["202208",14.12],["202207",13.71],["202206",13.49],["202205",13.34],["202204",13.29],["202203",13.01],["202202",12.48],["202201",12.45],["202112",12.56],["202111",12.78],["202110",12.54],["202109",12.26],["202108",11.94],["202107",11.73],["202106",12.04],["202105",12.09],["202104",12.08],["202103",11.7],["202102",12.69],["202101",11.35],["202012",11.59],["202011",11.93],["202010",11.85],["202009",11.77],["202008",11.5],["202007",11.59],["202006",11.76],["202005",11.79],["202004",12.04],["202003",11.81],["202002",11.72],["202001",11.46],["201912",11.71],["201911",11.99],["201910",11.81],["201909",11.81],["201908",11.76],["201907",11.71],["201906",11.8],["201905",12.01],["201904",12],["201903",11.6],["201902",11.6],["201901",11.44],["201812",11],["201811",11.6],["201810",11.5],["201809",11.39],["201808",11.27],["201807",11.09],["201806",11.2],["201805",11.36],["201804",11.39],["201803",11.35],["201802",10.87],["201801",10.63],["201712",10.89],["201711",11.23],["201710",11.09],["201709",11.12],["201708",10.95],["201707",10.87],["201706",11.03],["201705",11.05],["201704",11.09],["201703",11.12],["201702",11.16],["201701",10.74],["201612",10.85],["201611",11.1],["201610",11.02],["201609",11.11],["201608",10.91],["201607",10.83],["201606",10.95],["201605",11.1],["201604",11.26],["201603",11.19],["201602",10.99],["201601",10.88],["201512",11.19],["201511",11.38],["201510",11.35],["201509",11.51],["201508",11.43],["201507",11.5],["201506",11.78],["201505",12.04],["201504",12.1],["201503",11.56],["201502",11.62],["201501",11.49],["201412",11.81],["201411",11.98],["201410",12.07],["201409",12.03],["201408",12.05],["201407",12.05],["201406",12.17],["201405",11.93],["201404",12.06],["201403",11.75],["201402",11.25],["201401",11.22],["201312",11.27],["201311",11.67],["201310",11.68],["201309",11.36],["201308",11.45],["201307",11.41],["201306",11.44],["201305",11.57],["201304",11.43],["201303",11.03],["201302",11.05],["201301",10.84],["201212",10.99],["201211",11.11],["201210",11.16],["201209",11.11],["201208",10.96],["201207",10.87],["201206",10.97],["201205",10.84],["201204",11.02],["201203",11.01],["201202",11.05],["201201",10.77],["201112",10.89],["201111",11.19],["201110",11.23],["201109",11.16],["201108",11.15],["201107",11.16],["201106",11.25],["201105",11.27],["201104",11.16],["201103",11.13],["201102",10.65],["201101",10.7],["201012",11.25],["201011",11.32],["201010",11.62],["201009",11.58],["201008",11.7],["201007",11.71],["201006",11.85],["201005",11.94],["201004",12.12],["201003",11.51],["201002",11.34],["201001",11.21],["200912",11.52],["200911",12.08],["200910",11.97],["200909",12.05],["200908",12.15],["200907",12.38],["200906",12.66],["200905",12.84],["200904",12.88],["200903",12.93],["200902",12.84],["200901",12.6],["200812",13.08],["200811",13.28],["200810",13.53],["200809",13.51],["200808",13.84],["200807",14.06],["200806",13.43],["200805",12.94],["200804",12.53],["200803",11.97],["200802",11.35],["200801",11.65],["200712",11.75],["200711",12.03],["200710",12.42],["200709",12.44],["200708",12.4],["200707",12.51],["200706",12.73],["200705",12.62],["200704",12.56],["200703",12.46],["200702",11.97],["200701",12.08],["200612",11.91],["200611",12.52],["200610",13.18],["200609",13.25],["200608",13.35],["200607",13.31],["200606",13.41],["200605",12.98],["200604",12.72],["200603",12.42],["200602",12.12],["200601",11.91],["200512",11.42],["200511",11.5],["200510",11.6],["200509",11.66],["200508",11.47],["200507",11.29],["200506",11.21],["200505",10.72],["200504",10.15],["200503",10.05],["200502",9.5],["200501",9.15],["200412",9.24],["200411",9.37],["200410",9.57],["200409",10.32],["200408",10.71],["200407",10.48],["200406",10.44],["200405",9.76],["200404",9.39],["200403",8.93],["200402",8.55],["200401",8.61],["200312",8.38],["200311",8.59],["200310",8.93],["200309",9.52],["200308",9.89],["200307",9.97],["200306",10.07],["200305",9.35],["200304",9.03],["200303",9.02],["200302",8.11],["200301",7.71],["200212",8.07],["200211",7.44],["200210",7.51],["200209",8.04],["200208",8.07],["200207",8.49],["200206",8.5],["200205",8.15],["200204",7.98],["200203",7.79],["200202",7.98],["200201",8.11],["200112",8.23],["200111",8.4],["200110",9.11],["200109",9.27],["200108",9.53],["200107",9.55],["200106",9.63],["200105",8.91],["200104",8.52],["200103",8.39],["200102",8.02],["200101",7.68]]}]
They have more utility solar than California as of last year in absolute terms, they're second if you include distributed behind the meter solar which California has more of.
Texas also lead in absolute amounts of wind, gas and coal. They have a lot of generation.
That makes sense. It's hard to appreciate the scale of these things without understanding that Texas, although the less populous of the two states, uses far more energy, and in particular far more grid electricity, because of their climate, the energy intensity of their economy, their ideological opposition to energy efficiency mandates, and their relative lack of small-scale, distributed, behind-the-meter generation and storage.
Also the little sound bite about peak demand, Texas has enormous capacity during the summer as well; far, far exceeding demand. It makes total sense to be testing and connecting multi-GW consumers during these months.
> Also the little sound bite about peak demand, Texas has enormous capacity during the summer as well; far, far exceeding demand. It makes total sense to be testing and connecting multi-GW consumers during these months.
Texan here. Actually, it depends on how hot our summer weather gets and, thus, how much A/C use is in play. When we get into a not-unusual run of multiple 100-degree (F.) days in a row, the available capacity often drops to the point where ERCOT begins issuing alerts about things like suggested times of day when running certain appliances might not be wise. Having lived through the nightmare of the February '21 winter storm outages, I keep the ERCOT dashboard[0] as a bookmark and check it at least once, every day of the year.
I’m curious how this works for other large consumers. Do they have some kind of artificial load that lets them gradually reduce consumption instead of doing it all at once?
If you're large enough your connection to the grid is a negotiation with an engineering team.
The utility will force you to put equipment to correct for power factor (massive capacitor bank), resistive load, etc.
The utility also charges commercial users for apparent power (includes reactive power, or that sloshes around setting up a steady state), as opposed to just real power charged to residential users.
EDIT: in case your wondering, yes resistor loads is just glorified bunch of short circuits and a fan.
With the obligatory, must see, BBC Britain from Above segment in which Andrew Marr fly on the wall narrates the most challenging part of a live load engineer (UK)'s day ... <doof, doof, doof> the East Enders kettle break.
Pretty much. Except the law of large numbers breaks down in correlated events.
Like a black out.
What really hurts is are all the rotating machines, especially the one-phase ones (fridge, AC, blower). Those have transients that last several cycles and are electrical shorts until the back emf is setup.
Utilities will try to roll the power back on in sections to avoid instability, starting with hospitals and those near hydro plants.
But you can help put by turning off your stuff during the blackout and when the light come back on walking around the house to turn them back on.
A single ~50MW facility dropping off the grid suddenly is something the grid can absorb fine.
The problem is with facility-level grid-responsive safety mechanisms. If the grid sags a bit and several DCs ~simultaneously disconnect, you have a problem. If the DCs going offline causes further instability that causes other facilities to disconnect or grid protection breakers to open, you have a cascading chain reaction
ERCOT operates under a “connect and manage” interconnection philosophy. This means things can actually get built quickly (unlike other ISOs), but some management is required.
Of course, there are some downsides to this approach, but overall I think it’s a superior. Just look at renewable and battery penetration compared to other ISOs.
The problem: multi-GW DCs going suddenly offline leading to rapid transient overproduction.
What's missing is sufficient FFR and M-FFR BESS (battery) capacity to smooth out these sudden disconnects. There's only about 20 GW of BESS generation capacity total in the system now, and ERCOT needs at least 3x more. Placing large FFR and M-FFR near these giga DCs is what it will take to absorb transient overproduction transients from hurting generators or tripping the grid. This is something DCs should pay for rather than upending how the grid works for problems they cause.
I have read in other contexts that Texas has the least stable power grid in America. It has more downtime than even California, with it's annual wildfires. Despite the regulatory appeal, it does argue against data center expansion in Texas generally.
A lot would probably be an understatement. Aren't most batteries in data centers designed to just hold load for seconds or a few minutes at a time while generators start up?
If the problem is an instant disconnect of a large load, switching that large load into charging batteries at that instant, for a minute or so, feels like a solution.
Why is this confusing? Batteries are both generator and load. If the grid needs load to stabilize frequency and voltage, batteries can become loads instantly.
I'd rather see legislation banning crypto mining and AI data centers from the public grid entirely. No sense in forcing the broader public to subsidize them.
The problem with that is one of the best things we have to control pollution at power plants is the rules that go into place when connecting to the US grid (I know TX is different).
I really don’t want to incentivize private power plants that aren’t on grid. Or just running tons of industrial sized generators instead.
If we’re going to allow enough of this stuff to be built that it can destabilize things why not require they behave and don’t stop off like that? Some sort of organized draw down?
And if they don’t? Mandatory cutoff for X amount of time. Weeks/months.
Right, that’s what I was thinking of when I wrote my comment. Regulate back. If there is no will to do so, well, that’s a choice. Write the law, pass the law, aggressively enforce civil and criminal penalties for violations. They haul gas generators in without a license? Confiscate and tear them down for scrap (which will be painful, as these turbines are in short supply and their manufacturers are backlogged years into the future), in accordance with law you pass. Hold the utility liable if they provide a fossil gas pipeline connection. Humans like Elon may not care, but utilities have something to lose. Find “one throat to choke” as the saying goes.
It is not politically easy, but it is logistically straightforward.
The general problem with this approach is that it's not possible for the legislature to simultaneously be deliberative and fast-acting.
If you pass a nuance-free emphatic rule that says no fossil fuel generators, does that include the backup generators at a hospital? If it does and then the hospital loses power, that's not just a political problem, that's a "people will die" problem. But if it doesn't then you're going to turn around and find that Elon Musk's data center is hosting some hospital's IT system and then running the generators (that happen to also power the rest of the entire facility) to keep it online. Time will pass before this loophole can be detected, more time until new rules can be promulgated, and then they'll find a different loophole and the process begins again.
That process tends to make people frustrated and then they want to abandon the rule of law. Stop having rules that say what you can't do and just have rules that say you can't do anything and then selectively prosecute the people you don't like. The modern system has been evolving to work more like that, but that's how you enable people like Trump. Making the system work like that is a disaster.
What you need to do is find a better way to solve the problem in general, like a carbon tax, rather than trying to play whack a mole with overly-specific rules until there is such a thicket of them that you're really playing "show me the man and I'll show you the crime" -- or setting things up for someone else to.
If they weren't on the public grid, they would just slap in a bunch of gas turbines and run one of the noisier, more polluting sources of electricity. I think it would be better if we required them to replace the power they used, but do so on the grid so that it benefits everyone.
The newest one that I know of will use gas turbines as generators and will connect to the grid. [0] It is sited conveniently to operating natural gas compressor stations out in the Barnett Shale field and near to a local lake for water. That lake is effectively a deep mud-hole that has an agreement with a larger regional lake to pipe water from the larger lake to the local lake to help meet local demand.
If you ever visited the town and hit the restaurants during the summer you get a nice taste of the lake bottom when it flips. Pretty nasty yet few restaurants use water filtration to deliver fresh-tasting tea and water to their customers.
That's OT though.
The county will have a special meeting on data centers this Tuesday, June 9 where they are seeking public input. I expect that there will be plenty of people and hope they have time to give people an opportunity to speak. I visit this county regularly for grocery shopping, fuel, etc and if you go south one county there are already plenty of people who are sick and tired of the bitcoin operations and how they have disrupted life in that county. I hope we have a good turnout for this meeting so that this new operation gets canned before they build anything.
Here is a quick overview of the proposed site. [1] You can see in yellow the 200 acre section where turbine generators will be sited. In the light blue area adjacent you can see a small part of the larger parcel where the data centers will be built. This is part of one of the last large ranches in this part of the county. The others are actively being developed for large (>50000 residents) developments that were supposed to be self-contained communities but which have evolved to be large residential areas that dump traffic onto undersized freeways which are currently under construction to handle the huge volume of new traffic and traffic from all the other losers who will live out there.
The red polygon show existing grid substation intertie where high tension power lines converge or radiate out across the countryside. The light blue polygon is an active natural gas production and compressor station. There is more natural gas infrastructure just below the red polygon too. The lake is obvious in the lower left corner. The natural gas wells and compressor station date to early Barnett Shale production in the county, before 2008. The electrical substation is legacy and has been expanded more than once during the last 15 years.
The black polygon is a small rural subdivision of 8 ~2.2 acre ranchettes around one original ~4.2 acre home site. Those people bought a small plot of ranchland for their own slice of rural Texas. That section was subdivided into lots about 9 years ago so every one of those people bought homes next to an operational gas compressor station. I wonder whether they will be at the meeting.
I'd rather see legislation banning everyone from the grid. Why have a common resource if its no longer common. Just get rid of it. Ban everyone from using the roads, who knows, they might be transporting evil computer hardware to an evil data centre using a road. Farms obviously have to go, they supply food to people who write code that gets run in datacentres. In fact, just ban all trade and commerce to be on the safe side*
*The US should implement this policy for real for my personal amusement.
I think it's time to put data centers on a power budget. If they want to make more money, they need to become more efficient and eliminate AI fraud, waste, and abuse.
For those who dont know why this is important:
The loads are slowing down the generators that are burning a well metered amount of fuel to stay at 60Hz. This is a delicate balance since the phase angle must also be spot on.
If a generator and the local line disagree on f, phase or V, you have a short circuit.
If you lose a large amount of load, your generator will spin up with the excess fuel until the control system re-establishes the right amount of fuel.
But now your generators are out of sync! No worry, for small disturbances the dissipative losses sync everything up like syncros on a manual transmission.
But the disturance cant be too big!
Rotating machines are big and heavy, so the first line of defense is their inertia. But this is a finite (and precious) resource.
Contrary to belief, renewables, or generally speaking DC, makes things this stability problem worse. They generate large amounts of power while providing no inertia.
You'd think it isn't a big deal since the DC-AC converter can just synthesize whatever is needed. Heck just keep it rigid at 60 Hz with no phase change.
Well the later doesn't work - the rest of the grid is no longer at that phase and frequency so you got yourself a short.
Furthermore, the DC-AC converter, despite their manufacturers' promise, has no good way to establish what f and phase it should be at during a disturbance (and these magic codes are closed source, believe it or not)
Anywho, a large enough loss of load causes the grid to enters into unstable oscillations, causing protective relays to trip causing a zipper effect where the grid goes down.
Now restart will take a few days depending on the energy mix (fastest for hydro heavy)
Long story short - this is not a trivial problem, and the data-centers can't be allowed to just dump load willy nilly.
EDIT: made it clear that the grid killing disturbance is not caused by renewables; not exclusively anyway. Everyone has to play nice or the grid goes down.
Grid forming batteries and inverters are a thing. They can control the frequency on the grid. Just a matter of getting the right equipment.
The nice thing with data centers is that they are somewhat flexible. It's not a constant load. Data center operators can choose to reduce load. And if properly engineered, they could do so automatically based on signals from the grid.
The issue with outdated grids is that it relies on technology (spinning mass) that's at this point a century old. Which makes it brittle against outages like you describe. The solution is not more spinning mass but batteries and renewables to take the place of that spinning mass. A battery can respond to oscillations in milliseconds. If you then add flexible load that can spin up/down based on the amount of available power, you gain a lot of stability.
> The issue with outdated grids is that it relies on technology (spinning mass) that's at this point a century old. Which makes it brittle against outages like you describe. The solution is not more spinning mass but batteries and renewables to take the place of that spinning mass. A battery can respond to oscillations in milliseconds. If you then add flexible load that can spin up/down based on the amount of available power, you gain a lot of stability.
Why the spinning mass technology being "century old" (more like "millennia old" but anyways) is a problem somehow? The Newton's First Law didn't change much in the time that has passed.
Anyone who tried to "respond to oscillations in milliseconds" knows how hard that problem is because the force you apply is integrated twice before it takes effect. Try stabilizing a swing by pushing it's forward when it's behind the equilibrium point and pushing it backward when it's ahead of it. Now imagine a grid of swings connected by rubber bands and a distributed system of independent actors responding to oscillations. There are much more ways in which this system can diverge rather then converge.
Time may prove me wrong, but the arguments like "spinning mass is old therefore should be replaced" certainly won't.
> Newton's First Law didn't change much in the time that has passed.
No, but technology has moved on quite a bit. Heavy fly wheels are no longer the state of the art here.
A few tens of GW of battery capacity (i.e. a few dozen nuclear plants worth of capacity) that can switch on/off in milliseconds can do a lot for grid stability. That's part of the reason why grid operators are rolling out so much batteries. It's not necessarily about supplying energy for a very long time but about smoothing out peaks and dips in energy supply and demand and responding more or less in real time to that.
This stuff is basically being rolled out at industrial scale in a lot of places. Australia, China, etc. pretty much run increasingly on mostly renewables. This is no longer as speculative as it would have been ten years ago.
Yes, there are engineering challenges with rolling that stuff out in a lot of places. And even more policy and regulation challenges. Actually that is, by far, the #1 challenge in places like the US and Europe. Grid operators are simply structured and incentivized wrong to deal with this stuff efficiently. Texas is actually not doing too bad relative to e.g. California. But they clearly have some challenges still.
> A few tens of GW of battery capacity (i.e. a few dozen nuclear plants worth of capacity) that can switch on/off in milliseconds can do a lot for grid stability.
I think this is a misunderstanding of the problem.
Now, don't get me wrong, I believe flywheels are no longer involved but at one point they were for batteries and solar. Not to store energy, but rather to form the output voltage and to give the correct "inertia" of the waveform to maintain a correct phase with the grid. Prior, both batteries and solar were frequency followers. They'd look at the previous peaks and valleys to determine what their output voltage (or resistance) should be. If the input voltage fell too far, both solar and batteries would cut off to avoid damaging equipment on the grid. This is part of why the winter storm killed the texas grid (to my understanding) the voltage dipped too low which ultimately caused renewables to shut off completely to avoid damaging the grid. That all was somewhat of a cascading disaster.
Flywheels have been used as an inertia source to allow for both solar and batteries to act more like a hydro or fossil fuel generator. That's the grid forming technology. I believe (can someone verify?) that there are now all digital versions of this. But it's delicate software. Getting it wrong can do really bad things like destroying other generators or breaking expensive fuses.
I would imagine that any single renewable plant could handle that - the the frequency drifts out of sync, it's only the inverter that will blow up.
It's an expensive part to be sure, but not so expensive that a plant couldn't keep a few spares.
Respectfully, I believe you have confused "new" with "state of the art". China is likely using batteries because their battery production is subsidized, Australia's trading partner is China so basically equivalent.
Perhaps a bad analogy, but it seems these battery systems attempt to stabilize by "pushing", whereas spinning mass can also work through "dragging" the phase. So eventually, you will have just a few rotating masses setting the freq and phase, with more and more systems tracking that and pushing, which seems like a recipe for chaotic equilibrium.
No, they are doing that because they have a ten year head start on the rest of the world. The US & EU not having done their homework is entirely on them. The US is choosing to pump a lot of funding into fossil fuels instead. You could say that China is simply spending smarter.
The rest is just economics. We might suck at making batteries. But we suck even more at making new gas/coal plants or fueling those cost effectively.
Anyway, this stuff is being deployed by the hundreds of gwh per year now. Much of it in China but some non trivial amounts in places like Texas and California as well. As a result, the grid is actually getting more stable, not less stable.
> Respectfully, I believe you have confused "new" with "state of the art". China is likely using batteries because their battery production is subsidized, Australia's trading partner is China so basically equivalent.
This is comically wrong. You're just making things up. Australia is at the forefront of this new tech due to necessity and some good luck. Scroll to the bottom of this article and have a look at the graph. It shows Australia has 5 times the grid forming battery infrastructure deployed or under construction than China or the US, or pretty much any other country. It is very much SOTA, needed for a grid that is a rapidly changing mix of rooftop solar, hydro, coal, gas and wind spread over a country the size of the continental US. Rotating masses are not going to cut it.
https://arena.gov.au/blog/australias-grid-forming-battery-re...
> nuclear plants worth of capacity) that can switch on/off in milliseconds
Nuclear has this little problem getting in the way of switching on/off quickly:
https://en.wikipedia.org/wiki/Xenon-135
Yes we know the solution is infrastructure. The question is who gets to pay for it.
In America that seems to be the dying small town whose only economic value is cheap land.
Infrastructure is an investment with an ROI. Fuel is an expense that delivers energy once and that is never recovered. The only relevant debate here is on how to finance the relative attractive investment for renewables and over how long it delivers that ROI.
That varies per region but this is a sector that is now about 2x the level of investment for fossil fuel related infrastructure. A lot of which is increasingly looking like the ROI might never happen.
> over how long it delivers that ROI
In the US, this is moot if that time period is more than 3 months.
To add on to this overview, there are a new class of inverters in research called grid forming inverters which don't just follow the rest of the grid. One interesting technique is they can simulate the inertia of a traditional rotating machine/generator.
https://iten.ieee-ies.org/journal-featured-article/2025/grid...
10+ MW voltage-source converters that can't do up to around 80% of their nominal capacity as mostly-reactive apparent power with stabilizing synthetic inertia scaled as desired/specified are a mostly software issue, stemming from lack of regulatory pressure incentivizing the engineering complexity of that.
Though if you want to do a smoothing action on real power flux you'll have to colocate battery capacity with the converter. Which to be clear is fairly cheap to do as long as you get compensated for the substantial frequency stabilization capacity this represents. I'm talking like 15~120 minutes at converter nominal AC power of battery capacity.
The first 10~20% of reactive power are almost free from the converter electronics, btw....
The software is being written along with the regulations and compensations and it's being rolled out in the real world, just as you described.
That’s a simplified and somewhat outdated version, there’s a huge range of technology to mitigate each of these issues but infrastructure generally isn’t free. A turbine does provides inertia for free where you need to pay for a flywheel or battery system.
In the end a renewable heavy grid can be extremely resilient due to all the batteries, but smart battery systems need to be incentivize or mandated because ‘dumb’ batteries are cheaper.
The ELI5 version is you're arm-wrestling someone and they suddenly let their arm go limp, so your arm slams down on the table since you can't react that fast.
To build on this analogy, tug of war fits a bit better. Nothing dramatic happens if one person let go, but if half of one team just let loose at the same time without communication, bad things happen.
I think an even better example is what happened to the Tacoma Bridge. You have an oscillating frequency and phase that needs to be precisely tuned in order for everything to work well. If something is out of sync wrong, you can end up with higher peaks, lower valleys, or flatter locations which can ultimately cause catastrophic failure.
In the Tacoma example, the input frequency continued to add onto the bridges motion which ultimately caused it to destroy itself. In an electric grid, a misaligned phase can cause excessive spikes (imagine 480V when you expect 240V) or the generators to ultimately burn themselves out because 2 generators are fighting with each other, one trying to raise the voltage while another is trying to decrease the voltage. The really tricky thing is that load (particularly inductive or capacitive loads) look almost exactly like an unaligned generator.
https://www.youtube.com/watch?v=XggxeuFDaDU
Reminded me of this fun video of a guy spinning up a water turbine generator and getting it synchronized with to the grid the old-style needle phase meter, then connecting it successfully:
https://www.youtube.com/watch?v=xGQxSJmadm0
I interpreted from that EU grid post mortem that individual generators are coordinated using out of band comms channels that aren’t the power grid itself. Am I mistaken that they do this? Is it done there, but absent here?
Keeping the build trend going:
Communication that detects the release and travels as close as possible to the speed of the natural signal in the rope, and is robust enough to recover without losing stability if the other team grabs the rope again while your "let go" signal is mid-flight.
One way to dampen this is to put a really strong guy on each side, with instructions to never let go of the rope. These are the flywheels of the grid.
Thanks. This is a great analogy.
That's not true. Modern batteries and inverters can be really good at providing synthetic inertia.
Proof by example: https://spectrum.ieee.org/grid-scale-battery-scotland
As the article says, this is difficult and adds significantly to the cost. But it is very possible.
(It also adds significantly to the cost of spinning mass generators too. Delivering 250% of nominal current in a short circuit situation requires beefing up a lot of components).
There are many others.
Actually large data centers at least if done in a vaguely alirack style architecture, can do this with a decent fraction of their nominal power for very little hardware cost, as reactive power and real power add up via Pythagoras (`apparent=sqrt(real^2 + reactive^2)`) to the apparent power (rms voltage times rms current, which is what the 60Hz electronics and 60Hz transformers care about). The first 10-ish % are nearly free.
And alirack style datacenters have large 3-phase converters between the grid and some 240 (nowadays often 350) V DC bus, with the battery banks directly (with just fuses and sometimes a little bit of balancing/nudging power (think 10% of battery power rating)) on the bus, and then the servers also directly consuming from that bus.
The large converters on the battery bus thus allow synthetically smoothing load transients to the grid using the batteries to smooth that power draw. This has just minor additional wear on the batteries and a small power efficiency impact from hitting through the batteries, both of which are easily paid by anything market-rate of providing that grid service. Because they already need the power electronics and batteries anyways, unlike a utility battery farm that at best can argue day/night load shifting of solar production as the reason for the electronics and batteries to exist.
In that same spirit it's also effective to put batteries on the DC bus (between MPPT and inverters) of large solar farms, because they need the electronics anyways and it's actually reducing the required inverter&transformer capacity of the solar farm by peak-shaving.
Would demanding that large spikey users of energy like data centers implement some sort of demand ramping/isolation from the grid in the form of a massive capacitor bank or flywheel generator between them and the grid help reduce the risk here?
The data centres aren't inherently spikey, in general use their consumption is reasonably predictable.
However, if a DC detects that the _grid_ is wobbly (voltage or frequency deviations) the DC will disconnect without warning, and switch to its batteries and generators.
The grid complains because it's suddenly lost hundreds of MW of load. For the DC to have isolation capability, it would need a load-sink which can consume roughly the same power as the DC in normal operation, and can take in that load at a moment's notice.
It's a hard problem to solve, and probably better managed at grid-level than DC-level.
> The grid complains because it's suddenly lost hundreds of MW of load. For the DC to have isolation capability, it would need a load-sink which can consume roughly the same power as the DC in normal operation, and can take in that load at a moment's notice.
Thats why always have all the EVs be connected all the time (except the 20 mins they drive). EVs can provide demand as a service and take excess power whenever its available (instead of solar curtailment), and also provide an immediate source of load when events like this happen. Its a shame US is anti EV, it has the best systems at scale that can be leveraged to transform the entire energy ecosystem.
My intuition is that there would be a fairly stable base load, but doing something like switching on a new training run of a frontier model would be incredibly spiky, thousands of GPUs going from somewhat idle to 100% in seconds.
This is where a smart grid would really shine, because the utility could drop the electricity price to 0 or even negative and turn on 100,000 air conditioners/heaters to effectively give people free climate control to absorb the load imbalance.
Minor note: when discussing electricity, abbreviating datacenter as “DC” is confusing
Haha good spot. And when the data centre is running on batteries, the DC uses DC. And if it's in the USA capital, it could be a DC using DC in DC.
>it would need a load-sink which can consume roughly the same power as the DC in normal operation, and can take in that load at a moment's notice.
Liquid rheostat. A big one.
> Contrary to belief, renewables, or generally speaking DC, makes things this stability problem worse.
Is there such belief? My feel is that anybody to whom electrical grid stability has even crossed their minds know this.
Some of us in Texas are all too familiar with the problem of balancing load with generation, the risk of a cascade failure causing a slow restart.
During winter storm Uri, they did a duty cycle where we only had power available for ~6-12 hours at a time on the days it was available. This was apparently to avoid that very problem.
So far as I know, the obvious mitigations like winterizing NG generation and/or peering with neighbor grids have not been performed.
Practical Engineering made an outstanding video on this: "What Really Happened During the Texas Power Grid Outage?" https://www.youtube.com/watch?v=08mwXICY4JM
I've often thought that the target* grid phase should be encoded into a high frequency signal, say at 50kHz. Generators without inertia can immediately switch to the required phase and inertial systems can work towards it or disconnect their outputs if they stray too far.
The problem in my mind is that, as it stands, the signal that everyone has to latch on to is itself affected by load and by different generators latching on with different time constants in a complicated feedback loop. Would having a single authoritative source be an improvement? Would that be a way to eliminate the need for inertia?
*I suppose the "target" phase would probably be based primarily on the output from the biggest inertial systems and take into account their ability to adjust phase and frequency.
Synthetic inertia has come a long way. An inverter knows just as well as a rotating generator what frequency and phase it should be generating, because it can mimic exactly the ideal equations of motion of a rotating generator with programmable inertia.
When you say "short circuit" - is this when wires get hit and make and burn, or this is reference to some other stuff?
Short circuit is when two sources try to drive the same node in a circuit at different voltages or frequencies. In an idealized circuit (such as where the wires are assumed to have zero resistance), current flow from one source to the other is infinite. In a real circuit, current flow rises to a level where "something has to give." This could include heating the wires between the sources or tripping some kind of protection mechanism.
I'm not great with analogies, but imagine a train with two locomotives, and each one is set to run at a different speed. There will probably be a lot of screeching of wheels etc. To make multiple locomotives work on a single train requires engineering them to be synchronized with one another.
Could the GPS provide a correct carrier wave for every electrical system to synchronize with?
Is it feasible to tie the grid frequency to an external clock reference, such as TAI?
Grid scale battery systems are also used to maintain proper synchronization.
So apparently grid forming or synthetic inertia can be provided electronically nowadays, but presumably it would not be too terrible to put a rotating mass in between the (e.g.) solar power source and the rest of the grid? So the electricity from the solar panels runs a motor that turns a generator. I suppose power is lost through friction and other inefficiencies, and you'd need to co-locate the solar panels into a big enough farm.
How about an across the board $1/W hook up fee for new customers? Thats about the price of installed capacity per watt.
New house with 200 A panel an assumed 30% utilization rate? $3600.
New data center with 80% utilization rates at 100MW? $80 million dollars.
New 10 GW data center? That'll be $8 billion.
It's outrageous that I'm paying an extra fee to export energy to a neighboring state to power a datacenter.
All projects have an interconnection fee.
Large projects will have a large load interconnection tariff that’s supposed to shoulder the costs of upgrading the infrastructure to support these new projects.
Data center discussions are weird right now because people assume these things don’t exist and propose them as solutions. They already exist.
The problem in the article is something different: The sites they’re talking about are designed to disconnect from the grid and use backup power when the voltage drops, which can be a problem because now there’s too much energy being supplied to the grid and not enough load to absorb it.
Isn't that fundamentally a coordination problem?
When voltage is dropping (or phase is falling out of alignment) don't you want to shed load to stabilize? The issue seems to be more that too much load could shed too quickly causing wild oscillations in grid conditions from an undervolt to an overvolt. Seems like the correct, and not terribly complicated, solution to this is to have a process whereby the grid operator can request large customers to shed load and move to backup power temporarily.
That seems like such a reasonable suggestion that I would be shocked if such a thing does not already exist and is simply just not reactive enough and too manual.
Texas has an independent energy grid, so are not exporting energy to them.
Yes, although, with enough datacentres, it starts to get linked to other grids via data i.e. it starts 'exporting' its energy via compute
Interesting when you start to take this insight seriously. You could imagine data centers becoming important military targets in the future if the US ever balkanizes. Beyond that, securing compute could become a part of everyday trade agreements.
As things industrialize further, a state's net compute flow might become a topic of campaign promises. We may later find correlations between states' access to quality compute and the educational/financial outcome of younger generations.
To be sure, my leverage and inertia with a $200/mo ChatGPT Pro subscription is already a lot higher than it is without. We're not talking about just chatbots anymore, modern agentic models represent a quantifiable increase in individual agency. Whether or not the job market implodes, I am able to make progress on things even while asleep or busy with something else. This kind of agentic capital has never been available to the working class before at scale.
States like Texas, with their independent grid, might find themselves in a favorable position in such a balkanization event, but past events have proved that ERCOT is not ready to operate a statewide power grid. I was in Texas during the '21 cold snap and it was brutal. I was using my gas stove for heat. If resources for compute suddenly became constrained, and compute was deemed a matter of security and economic survival, it would be a cutthroat environment for operators of Texas' existing 400+ data centers.
My rant was generally about what ought to be done to data centers.
In fact, I don't live near Texas. My state is having me finance a power line that will increase demand for our electricity.
Linear progression is unfair, just like for taxation. If you're going to make a law like this, it should be something like nlog(n) so that the big players that abuse the system pay more than the little guy.
Now you made me wish for nlog(n) income tax as well.
We used to have something similar in France: solidarity tax on fortune (ISF), abolished by Macron at the height of the gilets jaunes movement in a big "fuck you" move.
Now it's much easier for wealthy elites to not pay taxes at all, as outlined in the past year with the Zucman tax debates (which elites opposed for daring to propose a 2% floor tax rate for those who don't pay 2%), and lately with the ministry of Finance Amélie de Montchalin lying to the members of parliament about the existence of a note produced by her own ministry showing that over 13000 millionaires pay effectively 0% income tax.
n*ln(n) - a, such that below a, people receive a negative tax/progressive UBI.
> It's outrageous that I'm paying an extra fee to export energy to a neighboring state to power a datacenter.
I don’t find anything outrageous whatsoever about building out transmission to strengthen regional interconnections. Lord knows we are decades behind the curve on this. If AI datacenters are what finally lets us maybe start to get a tiny bit ahead of the game I’m all for it. When the bubble pops there will finally be a bit of pressure off, and perhaps we can get back to a tiny bit of overbuilt capacity to cover for exceptional events again. And more reasonable projects maybe won’t be sitting in a decade long interconnection queue.
At some point we get to all collectively pay for our parents lack of investment in energy infrastructure. We have been living off our (great?) grandparents investments into the future and wide scale deindustrialization of the economy since I’ve been alive. At some point you run out of inertia.
Could always move to Texas if you hate the idea of your state ever possibly spending a dollar that benefits someone across the border. They islanded their grid pretty much precisely due to this attitude.
Watching the PJM and MISO interconnections over the past decade or so has been illuminating. It’s been a very slow moving disaster in the making that nearly no one is paying attention to.
As for your proposed solution I doubt anyone - including the datacenter operators - would argue since that is largely what tends to happen as it is. Just change utilization factor to capacity factor and sounds good to me!
> MISO
As a retail electricity customer, I vastly prefer whatever is going on in their operational context.
I live 10 miles from the "border" between ERCOT and MISO and I've got a lot of experience with both, particularly in disaster scenarios. ERCOT is a total shitshow when it comes to the edges. The fuel mix in MISO is pretty terrible for the environment, but it is also unbelievably reliable. The Mississippi River is not to be underestimated in its logistical impacts. The price of natural gas drives the daily fuel mix proportions, but it doesn't affect the available base capacity and its financial status (fully amortized for over a generation). If natural gas gets expensive in ercot and the wind isn't blowing, there isn't anything else to give. MISO has 2-3x the coal and nuclear capacity that ERCOT does.
Your math on the house seems off by a factor of 4? But 30% utilization seems high, as that would be a $2k/mo electric bill at 20 cents/kWh.
Reframing what you're saying, that would be a connection fee that worked out to just under 12 cents per kWh used for the first year, which seems both somewhat reasonable but also probably not going to move the needle much on the large deployments?
I guessed what the utilization factor for a house was, purposely going for a very high number to show that it wouldn't add too much to the house.
Note, In proposing a one time fee, on the capacity to generate power, not the energy.
Yes, but using the 30% utilization factor I get 200A * 240V * 30% = 14.4kW , meaning a $14400 connection fee rather than $3600 ? $3600 would be a 7.5% utilization factor.
And yes, I got the one time fee aspect. My point is that if you're basing this off of expected utilization (rather than say the size of the service regardless of how much energy is used), and that expected utilization is roughly correct, then that one time fee can be thought of as amortized cost per unit energy over a specified amount of time. And that cost isn't even that high if amortized over a year.
No, because that doesn't go far enough and still shifts some costs onto everyone else.
Make the DCs themselves pay to play nice to work with the grid: BESS FFR/M-FFR, for generation capacity, and for transmission upgrades.
Right now, they're using privatized-deregulated communism to privatize the profits and socialize the costs. Everyone else shouldn't pay for these goddamn DCs.
I'm a bit confused about why this is a serious engineering problem.
If a gigawatt class DC suddenly needs to take its sensitive IT loads off grid, it could be designed with load banks on site to stand in for the IT load. These would be exceptional use only, so the specific cooling technology (obviously we want to boil water) is not much of an ecological concern. A gigawatt will vaporize ~100 gallons of water per second. How long until the grid can adapt? Five minutes? That's not exactly a heroic amount of water for these projects.
> could be designed with load banks on site to stand in for the IT load
Yes, but that's not what's happening. These giant loads are suddenly going offline and are making it the transmission and generation networks' problem. The DCs aren't going to pay for it when they can socialize the costs and problems onto others.
The most obvious answer is these big loads need FFR and M-FFR BESS (battery) capacity near them or on-site to decouple and absorb these huge transient load swings into something the grid can handle. ERCOT should set additional rules that large load customers must use load smoothing rather than simply dumping load at auto-reclosers/auto-cutouts. Not doing so forces the entire grid to be at risk of low inertia transient over-generation.
>why is[/]this a serious engineering problem[?]
Because the Texian grid-operator (ERCOT) is predominantly isolated from the Eastern & Western US grids [†], serving mostly-just Texans – this is to avoid US Federal regulations, by being an only-intrastate entity (technically, which is: the worst kind of "correct" [∑]). This is a socially-engineered "problem".
This means their needs for maintaining grid inertia (which frequency is a measure-er of) are more difficult, as Texas cannot just purchase any substantial/meaningful electricity – everything must be generated in-house.
Worse, the lack of Federal regulations means ERCOT/Texas can create their own rules/regulations/markets for the management of electrical infrastructure. Texas chose to make (IMHO) the most-purely crony-capitalistic market [for buy/sell of MWhours] in worldwide existance – which leads to decannual winterstorm Disasters (totaling hundreds of BBillion$ in damages / hundreds dead). The losses from wasted silicon-ingot depositions (alone) are staggering $,,,.00 – just another cost of doing business within Texas's jurisdiction [ß].
After 2021's storm, some "free market" customers were left with a week of billing costing more than the year's remainder!.
Abbott will just blame the windmills and not a cost-incentive fee-structure which does not incentivize cold-weather preventative maintainance. In fact, it does the opposite (since electricity can 25X price-increase). "Tell me the incentive and I'll tell you the outcome."
[∑] having grown up and been schooled primarily in Texas, I think the best review I ever agreed with was: "The [L]one Star State" – Texceptionalism will kill us all
[†] small DC-DC grid-ties (altogether a low, single-digit percentage of ERCOT's overall capacity) exist with both grids, as well as another smaller with Mexico
[ß] which is why most hospitals/datacenters have their own diesel/LNG generators, on-site – in Texas these have even less regulations because: "temporary" see (e.g.): Tesla's multiple Texas facilities' backupsystems – enormous point-source pollutors
----
If you want a good breakdown of the 2021 Texas storm's decimation of ERCOT/Texas/electricity, look for the great logs reddit /u/redditmudder did (in realtime) as this disaster was unfolding. I cannot remember exactly how low the frequency got... but I think we were closer to EU [50hz] than US [60hz] – as grid operators REAPED profits* from their failing infrastructure driving electricity orders-of-magnitude higher. This is all public information; it wasn't "windmills."
This almost seems like a straw man to me. Isn't the much larger problem the actual increased energy usage and making sure that all of this massive extra cost doesn't just get dumped on consumers?
I am a huge proponent of AI actually, but very suspicious that "financiers" are suddenly creating what amounts to an energy tax by finding legal ways to sneak extra fees or rates into our electricity bills to cover build out and commercial usage costs.
But as far as smoothing out demand, my (admittedly a layperson) theory is that we need to force them to adapt more solar and wind and at the same time more facilities for handling the variable production from that. Such as more large batteries and a shift to large scale long term storage of renewable fuel like hydrogen or other fuels produced directly from renewable sources.
If you have a large production and storage of renewable fuel, then maybe you can build that in such a way that it can handle significant input variations that could include excess grid power.
Solar and wind would make it worse.
What matters is peak capacity. Using the grid as a free battery for when your intermittent sources of energy go away against your fairly constant loads only makes the grid weaker overall. The best part of these huge predictable load centers is utilization factor.
Storage of some sort would probably help some, but overall the best type of load for a grid is predictable constant usage. Bonus points if it can reasonably be part of a load shedding/demand response program.
Some kind of large aerostat / inflatabe chimney reaching up till the tropopause largely insulating for most of its length, but with heat engines on top and on bottom. I'm not proposing to use water / ice transition, but just consider a hypothetical strong suspension system, heaving a bucket of water weighing 1 mass unit can be powered by lowering a bucket of ice weighing 1 mass unit (ignore tiny density difference). So one can simultaneously bring 2 thermal baths close together, both at the bottom of the heat loop (where ambient temperature and tropopause temperature in the loop are close by each other) as well as at the top (where ambient cold tropopause temperature and surface temperature in the loop are close by each other). So both at the top and the bottom mechanical power can be generated with a heat engine.
The ground level vs tropopause temperature difference is not perfectly stable but largely maintained throughout day and night cycles, it is effectively base load, no nuclear required! We could generate energy while helping the planet cool.
Best to place these in seas, at least 1 structure length away from coastline, not in the middle of densely populated area. Salty sea water can be frozen (purifying it because ions get pushed into the brine) for desalination, brought to conventional thermal power plants, melted on their cold sides producing potable water while improving the thermodynamic efficiency of the thermal power plant (more electric energy per unit of fuel / rod spent).
Data centers are pretty useful for adding more renewables to the grid. They generally have both a UPS (for short-term outages) and backup generators (for long-term outages). UPS means they're already paying for inverters and some amount of batteries, and then in combination with a more renewable grid, it makes sense to put the grid storage batteries there, and they have the incentive to do it to take advantage of time of day metering by actually running on them during peak demand hours. Likewise, if you have a huge capacity backup generator and the grid is stressed because renewable generation is low (and therefore prices get high), you can turn it on. Which allows the grid to get more power from renewables the other 95% of the time.
Texas is the standing proof that "making sure that all of this massive extra cost doesn't just get dumped on consumers" is not a problem. Texas has the most load, the most marginal load added every year, and the cheapest retail power. Load grew 15% in the 5 years to 2024, and retail prices fell by 1¢.
And California is proof of the converse.
California's wholesale electricity prices are the lowest in the nation, half of what Texas' are.
But California's retail electricity prices are the second highest in the nation.
https://cleantechnica.com/2026/05/30/california-lowest-whole...
I don’t think that is true.
https://www.electricchoice.com/historical-electricity-pricin...
I am sorry to be the one to inform you about "inflation". The data you provided shows that rates increased from 2019-2024 at rates slightly below general CPI inflation.
Let's assume we're talking about core inflation (since headline inflation included changes in energy costs and that would hide the signal). Core inflation was us 23% from '19-24, but the energy costs in Texas were up 37%. Actually, headline inflation was up 25%, which is still below that set of price increases.
Please support your assertion that Texas energy costs decreased in that time period.
According to EIA State Energy Data System, "Average end-use energy price estimates by energy source, annual", electricity in Texas rose from $25.51 per million BTU (yes, I know) in 2019 to $28.95 in 2024, the most recent year available in the database. That's only 13.5%.
Ahh, I was focusing on residential energy (the cost borne by most of the people in Texas), which is up 25% per that source. The bulk and stability of industrial power is keeping prices down in aggregate.
[{"series_id":"ELEC.PRICE.TX-RES.M","name":"ELEC.PRICE.TX-RES.M, monthly","units":"cents per kilowatt-hour","f":"M","description":"Electricity sales to ultimate customer by state and sector (number of customers, average price, revenue, and megawatthours of sales). \n Sources: Forms EIA-826, EIA-861, EIA-861M","copyright":"None","source":"EIA, U.S. Energy Information Administration","start":"202603","end":"200101","data":[["202603",16.39],["202602",15.41],["202601",15.69],["202512",15.87],["202511",16.04],["202510",16.1],["202509",15.83],["202508",15.46],["202507",15.36],["202506",15.26],["202505",15.53],["202504",15.52],["202503",15.28],["202502",14.88],["202501",14.67],["202412",15.36],["202411",15.68],["202410",15.67],["202409",15.06],["202408",14.88],["202407",14.7],["202406",14.55],["202405",14.83],["202404",15.13],["202403",15.02],["202402",14.48],["202401",14.35],["202312",14.64],["202311",14.79],["202310",14.89],["202309",14.68],["202308",14.17],["202307",13.95],["202306",14.35],["202305",14.65],["202304",14.36],["202303",14.61],["202302",14.72],["202301",14.36],["202212",14.49],["202211",15.01],["202210",14.94],["202209",14.75],["202208",14.12],["202207",13.71],["202206",13.49],["202205",13.34],["202204",13.29],["202203",13.01],["202202",12.48],["202201",12.45],["202112",12.56],["202111",12.78],["202110",12.54],["202109",12.26],["202108",11.94],["202107",11.73],["202106",12.04],["202105",12.09],["202104",12.08],["202103",11.7],["202102",12.69],["202101",11.35],["202012",11.59],["202011",11.93],["202010",11.85],["202009",11.77],["202008",11.5],["202007",11.59],["202006",11.76],["202005",11.79],["202004",12.04],["202003",11.81],["202002",11.72],["202001",11.46],["201912",11.71],["201911",11.99],["201910",11.81],["201909",11.81],["201908",11.76],["201907",11.71],["201906",11.8],["201905",12.01],["201904",12],["201903",11.6],["201902",11.6],["201901",11.44],["201812",11],["201811",11.6],["201810",11.5],["201809",11.39],["201808",11.27],["201807",11.09],["201806",11.2],["201805",11.36],["201804",11.39],["201803",11.35],["201802",10.87],["201801",10.63],["201712",10.89],["201711",11.23],["201710",11.09],["201709",11.12],["201708",10.95],["201707",10.87],["201706",11.03],["201705",11.05],["201704",11.09],["201703",11.12],["201702",11.16],["201701",10.74],["201612",10.85],["201611",11.1],["201610",11.02],["201609",11.11],["201608",10.91],["201607",10.83],["201606",10.95],["201605",11.1],["201604",11.26],["201603",11.19],["201602",10.99],["201601",10.88],["201512",11.19],["201511",11.38],["201510",11.35],["201509",11.51],["201508",11.43],["201507",11.5],["201506",11.78],["201505",12.04],["201504",12.1],["201503",11.56],["201502",11.62],["201501",11.49],["201412",11.81],["201411",11.98],["201410",12.07],["201409",12.03],["201408",12.05],["201407",12.05],["201406",12.17],["201405",11.93],["201404",12.06],["201403",11.75],["201402",11.25],["201401",11.22],["201312",11.27],["201311",11.67],["201310",11.68],["201309",11.36],["201308",11.45],["201307",11.41],["201306",11.44],["201305",11.57],["201304",11.43],["201303",11.03],["201302",11.05],["201301",10.84],["201212",10.99],["201211",11.11],["201210",11.16],["201209",11.11],["201208",10.96],["201207",10.87],["201206",10.97],["201205",10.84],["201204",11.02],["201203",11.01],["201202",11.05],["201201",10.77],["201112",10.89],["201111",11.19],["201110",11.23],["201109",11.16],["201108",11.15],["201107",11.16],["201106",11.25],["201105",11.27],["201104",11.16],["201103",11.13],["201102",10.65],["201101",10.7],["201012",11.25],["201011",11.32],["201010",11.62],["201009",11.58],["201008",11.7],["201007",11.71],["201006",11.85],["201005",11.94],["201004",12.12],["201003",11.51],["201002",11.34],["201001",11.21],["200912",11.52],["200911",12.08],["200910",11.97],["200909",12.05],["200908",12.15],["200907",12.38],["200906",12.66],["200905",12.84],["200904",12.88],["200903",12.93],["200902",12.84],["200901",12.6],["200812",13.08],["200811",13.28],["200810",13.53],["200809",13.51],["200808",13.84],["200807",14.06],["200806",13.43],["200805",12.94],["200804",12.53],["200803",11.97],["200802",11.35],["200801",11.65],["200712",11.75],["200711",12.03],["200710",12.42],["200709",12.44],["200708",12.4],["200707",12.51],["200706",12.73],["200705",12.62],["200704",12.56],["200703",12.46],["200702",11.97],["200701",12.08],["200612",11.91],["200611",12.52],["200610",13.18],["200609",13.25],["200608",13.35],["200607",13.31],["200606",13.41],["200605",12.98],["200604",12.72],["200603",12.42],["200602",12.12],["200601",11.91],["200512",11.42],["200511",11.5],["200510",11.6],["200509",11.66],["200508",11.47],["200507",11.29],["200506",11.21],["200505",10.72],["200504",10.15],["200503",10.05],["200502",9.5],["200501",9.15],["200412",9.24],["200411",9.37],["200410",9.57],["200409",10.32],["200408",10.71],["200407",10.48],["200406",10.44],["200405",9.76],["200404",9.39],["200403",8.93],["200402",8.55],["200401",8.61],["200312",8.38],["200311",8.59],["200310",8.93],["200309",9.52],["200308",9.89],["200307",9.97],["200306",10.07],["200305",9.35],["200304",9.03],["200303",9.02],["200302",8.11],["200301",7.71],["200212",8.07],["200211",7.44],["200210",7.51],["200209",8.04],["200208",8.07],["200207",8.49],["200206",8.5],["200205",8.15],["200204",7.98],["200203",7.79],["200202",7.98],["200201",8.11],["200112",8.23],["200111",8.4],["200110",9.11],["200109",9.27],["200108",9.53],["200107",9.55],["200106",9.63],["200105",8.91],["200104",8.52],["200103",8.39],["200102",8.02],["200101",7.68]]}]
Do t they also have like the most solar?
Maybe in absolute terms? I haven't checked in a while. In terms of fraction of energy from solar Texas is still far behind California.
They have more utility solar than California as of last year in absolute terms, they're second if you include distributed behind the meter solar which California has more of.
Texas also lead in absolute amounts of wind, gas and coal. They have a lot of generation.
And with data centers (ai) they will not have enough
That makes sense. It's hard to appreciate the scale of these things without understanding that Texas, although the less populous of the two states, uses far more energy, and in particular far more grid electricity, because of their climate, the energy intensity of their economy, their ideological opposition to energy efficiency mandates, and their relative lack of small-scale, distributed, behind-the-meter generation and storage.
Taking note of the risk is important for everyone but shouldn't the follow up action be to improve the grid voltage stabilization mechanisms?
I have no idea of what entails but this seems more a quality of service issue (provided by the grid) and less about the consumers using the grid.
Charge them the cost of installing mitigations that can pick up the load if they drop it suddenly.
This all seems "routine".
Also the little sound bite about peak demand, Texas has enormous capacity during the summer as well; far, far exceeding demand. It makes total sense to be testing and connecting multi-GW consumers during these months.
> Also the little sound bite about peak demand, Texas has enormous capacity during the summer as well; far, far exceeding demand. It makes total sense to be testing and connecting multi-GW consumers during these months.
Texan here. Actually, it depends on how hot our summer weather gets and, thus, how much A/C use is in play. When we get into a not-unusual run of multiple 100-degree (F.) days in a row, the available capacity often drops to the point where ERCOT begins issuing alerts about things like suggested times of day when running certain appliances might not be wise. Having lived through the nightmare of the February '21 winter storm outages, I keep the ERCOT dashboard[0] as a bookmark and check it at least once, every day of the year.
[0]: https://www.ercot.com/gridmktinfo/dashboards
I’m curious how this works for other large consumers. Do they have some kind of artificial load that lets them gradually reduce consumption instead of doing it all at once?
If you're large enough your connection to the grid is a negotiation with an engineering team.
The utility will force you to put equipment to correct for power factor (massive capacitor bank), resistive load, etc.
The utility also charges commercial users for apparent power (includes reactive power, or that sloshes around setting up a steady state), as opposed to just real power charged to residential users.
EDIT: in case your wondering, yes resistor loads is just glorified bunch of short circuits and a fan.
That’s interesting. For the rest of us I guess it’s just the law if large numbers that our air conditioners don’t cycle off at the same time
In the UK we have that scenario with people turning on kettles during breaks in TV programs.
https://en.wikipedia.org/wiki/TV_pickup
With the obligatory, must see, BBC Britain from Above segment in which Andrew Marr fly on the wall narrates the most challenging part of a live load engineer (UK)'s day ... <doof, doof, doof> the East Enders kettle break.
* https://www.youtube.com/watch?v=slDAvewWfrA
Pretty much. Except the law of large numbers breaks down in correlated events.
Like a black out.
What really hurts is are all the rotating machines, especially the one-phase ones (fridge, AC, blower). Those have transients that last several cycles and are electrical shorts until the back emf is setup.
Utilities will try to roll the power back on in sections to avoid instability, starting with hospitals and those near hydro plants.
But you can help put by turning off your stuff during the blackout and when the light come back on walking around the house to turn them back on.
No. I imagine they are less sensitive and generally stay online and are only de-energized if the utility has an outage
But they can have their own internal fault and have to shutdown.
A single ~50MW facility dropping off the grid suddenly is something the grid can absorb fine.
The problem is with facility-level grid-responsive safety mechanisms. If the grid sags a bit and several DCs ~simultaneously disconnect, you have a problem. If the DCs going offline causes further instability that causes other facilities to disconnect or grid protection breakers to open, you have a cascading chain reaction
It always seems wild to me how, in the US, something like the grid is just a wild west.
"ERCOT said it is reviewing the test failures and drawing up plans to protect the grid from disruptions."
Why are they even allowed to connect? / Why are they not kicked off? / Why aren't they being forced to add their own grid inertia?
ERCOT operates under a “connect and manage” interconnection philosophy. This means things can actually get built quickly (unlike other ISOs), but some management is required.
Of course, there are some downsides to this approach, but overall I think it’s a superior. Just look at renewable and battery penetration compared to other ISOs.
This isn't the US - it's just the state of Texas. The rest of the contiguous United States is on two large grids. Texas has its own.
It's up to ERCOT to figure their own mess out. They will. Nobody else really has to care about it.
They aren’t being allowed to connect until they get it worked out. At least that’s how I read it.
Do we really need to keep slamming the grid and killing the planet for pseudonymous casino chips?
Can someone please merge crypto with llm training/inference somehow?
The problem: multi-GW DCs going suddenly offline leading to rapid transient overproduction.
What's missing is sufficient FFR and M-FFR BESS (battery) capacity to smooth out these sudden disconnects. There's only about 20 GW of BESS generation capacity total in the system now, and ERCOT needs at least 3x more. Placing large FFR and M-FFR near these giga DCs is what it will take to absorb transient overproduction transients from hurting generators or tripping the grid. This is something DCs should pay for rather than upending how the grid works for problems they cause.
I have read in other contexts that Texas has the least stable power grid in America. It has more downtime than even California, with it's annual wildfires. Despite the regulatory appeal, it does argue against data center expansion in Texas generally.
This sounds like a good reason to have a lot of batteries.
Texas was the leading state in new grid battery installs for the last few years.
A lot would probably be an understatement. Aren't most batteries in data centers designed to just hold load for seconds or a few minutes at a time while generators start up?
If the problem is an instant disconnect of a large load, switching that large load into charging batteries at that instant, for a minute or so, feels like a solution.
What?
Why is this confusing? Batteries are both generator and load. If the grid needs load to stabilize frequency and voltage, batteries can become loads instantly.
Oh, okay. I was confused because I was talking about batteries on the side of the data center.
Yeah, different batteries, different purposes.
Keeping the data center up is completely different from keeping the grid up.
Not only are the batteries too small; they're also on the wrong side of the disconnect.
I'd rather see legislation banning crypto mining and AI data centers from the public grid entirely. No sense in forcing the broader public to subsidize them.
The problem with that is one of the best things we have to control pollution at power plants is the rules that go into place when connecting to the US grid (I know TX is different).
I really don’t want to incentivize private power plants that aren’t on grid. Or just running tons of industrial sized generators instead.
If we’re going to allow enough of this stuff to be built that it can destabilize things why not require they behave and don’t stop off like that? Some sort of organized draw down?
And if they don’t? Mandatory cutoff for X amount of time. Weeks/months.
Ban private fossil generators above a certain size without a license. You can just do things. They can build as much solar and batteries as they want.
https://www.datacenterdynamics.com/en/news/google-confirms-1...
https://www.datacenterdynamics.com/en/news/google-announces-...
SpaceX is running a bunch of “portable”, high pollution gas generators in Memphis, TN specifically to get around the regulation you’re describing.
Elon definitely got the “you can just do things” memo.
The proposed regulation they're describing doesn't have that loophole.
It’s what I was thinking of when I said “lots of industrial generators” too, but I couldn’t remember where it was.
Right, that’s what I was thinking of when I wrote my comment. Regulate back. If there is no will to do so, well, that’s a choice. Write the law, pass the law, aggressively enforce civil and criminal penalties for violations. They haul gas generators in without a license? Confiscate and tear them down for scrap (which will be painful, as these turbines are in short supply and their manufacturers are backlogged years into the future), in accordance with law you pass. Hold the utility liable if they provide a fossil gas pipeline connection. Humans like Elon may not care, but utilities have something to lose. Find “one throat to choke” as the saying goes.
It is not politically easy, but it is logistically straightforward.
https://en.wiktionary.org/wiki/one_throat_to_choke
The general problem with this approach is that it's not possible for the legislature to simultaneously be deliberative and fast-acting.
If you pass a nuance-free emphatic rule that says no fossil fuel generators, does that include the backup generators at a hospital? If it does and then the hospital loses power, that's not just a political problem, that's a "people will die" problem. But if it doesn't then you're going to turn around and find that Elon Musk's data center is hosting some hospital's IT system and then running the generators (that happen to also power the rest of the entire facility) to keep it online. Time will pass before this loophole can be detected, more time until new rules can be promulgated, and then they'll find a different loophole and the process begins again.
That process tends to make people frustrated and then they want to abandon the rule of law. Stop having rules that say what you can't do and just have rules that say you can't do anything and then selectively prosecute the people you don't like. The modern system has been evolving to work more like that, but that's how you enable people like Trump. Making the system work like that is a disaster.
What you need to do is find a better way to solve the problem in general, like a carbon tax, rather than trying to play whack a mole with overly-specific rules until there is such a thicket of them that you're really playing "show me the man and I'll show you the crime" -- or setting things up for someone else to.
If they weren't on the public grid, they would just slap in a bunch of gas turbines and run one of the noisier, more polluting sources of electricity. I think it would be better if we required them to replace the power they used, but do so on the grid so that it benefits everyone.
The newest one that I know of will use gas turbines as generators and will connect to the grid. [0] It is sited conveniently to operating natural gas compressor stations out in the Barnett Shale field and near to a local lake for water. That lake is effectively a deep mud-hole that has an agreement with a larger regional lake to pipe water from the larger lake to the local lake to help meet local demand.
If you ever visited the town and hit the restaurants during the summer you get a nice taste of the lake bottom when it flips. Pretty nasty yet few restaurants use water filtration to deliver fresh-tasting tea and water to their customers.
That's OT though.
The county will have a special meeting on data centers this Tuesday, June 9 where they are seeking public input. I expect that there will be plenty of people and hope they have time to give people an opportunity to speak. I visit this county regularly for grocery shopping, fuel, etc and if you go south one county there are already plenty of people who are sick and tired of the bitcoin operations and how they have disrupted life in that county. I hope we have a good turnout for this meeting so that this new operation gets canned before they build anything.
[0]https://www.star-telegram.com/news/local/article315770666.ht...
Here is a quick overview of the proposed site. [1] You can see in yellow the 200 acre section where turbine generators will be sited. In the light blue area adjacent you can see a small part of the larger parcel where the data centers will be built. This is part of one of the last large ranches in this part of the county. The others are actively being developed for large (>50000 residents) developments that were supposed to be self-contained communities but which have evolved to be large residential areas that dump traffic onto undersized freeways which are currently under construction to handle the huge volume of new traffic and traffic from all the other losers who will live out there.
The red polygon show existing grid substation intertie where high tension power lines converge or radiate out across the countryside. The light blue polygon is an active natural gas production and compressor station. There is more natural gas infrastructure just below the red polygon too. The lake is obvious in the lower left corner. The natural gas wells and compressor station date to early Barnett Shale production in the county, before 2008. The electrical substation is legacy and has been expanded more than once during the last 15 years.
The black polygon is a small rural subdivision of 8 ~2.2 acre ranchettes around one original ~4.2 acre home site. Those people bought a small plot of ranchland for their own slice of rural Texas. That section was subdivided into lots about 9 years ago so every one of those people bought homes next to an operational gas compressor station. I wonder whether they will be at the meeting.
[1]https://i.postimg.cc/0NVn0ch9/datacenter.png
Didn’t Texas pretty much do the exact opposite thing recently?
I'd rather see legislation banning everyone from the grid. Why have a common resource if its no longer common. Just get rid of it. Ban everyone from using the roads, who knows, they might be transporting evil computer hardware to an evil data centre using a road. Farms obviously have to go, they supply food to people who write code that gets run in datacentres. In fact, just ban all trade and commerce to be on the safe side*
*The US should implement this policy for real for my personal amusement.
More scarce resources used to feed the surveillance state's fancy chat bots
I look forward to hearing from the usual suspects how the inevitable failure is the fault of wind and solar power.
I think it's time to put data centers on a power budget. If they want to make more money, they need to become more efficient and eliminate AI fraud, waste, and abuse.