I think we need to have a little more sympathy for GitHub. You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figured out, but a little sympathy goes a long way. I personally wish them the best and hope their on-call people can go back to getting normal amounts of sleep soon.
sandeepkd
I think somewhere down the line you have an answer to problem itself. Its for a while that companies are not into the business of sympathies, we should probably not get into that side for the topic.
1. Github has enterprise users who paid for the service, their day job requires Github to be available and working
2. Github has generous free tier which is the one which is exploring a lot more with the AI generated code.
It is a complexity in itself but the traffic should have been separated, the free users should not be allowed to bring down Github for enterprise customers (Just to clarify, I am free user myself). And if they do not have capacity it would have been perfectly fine to push back or throttle new users/repositories.
tyre
I agree that they should reserve capacity for enterprises and paying users.
But I think they’re in a tough spot. GitHub has historically been a huge supporter, proponent, and provider for open source projects. Engineers are difficult customers, to say the least, and the community would likely freak tf out of the segmented traffic.
The logical, pragmatic, and justifiable answer doesn’t always align with your market.
sandeepkd
Not sure if I would agree to that completely. Free tier is always ripe for abuse so in the beginning for a long time, Github has this requirement that you can only have a public repository if you are a non-paying user. Somewhere down the road when machine learning picked up (everyone was on the band wagon of hoarding the data, datalakes, hadoop, flink were at the top) they relaxed that condition allowing one to have a private repository on the free tier.
They are in a tough spot cause they want to continuously ingest all the data to train their LLMs. Any fork or decentralization at large scale of git is going to impact the training pipeline.
rand846633
Tbh, timing wise it appeared to me that GH started allowing private repos when one of the competitors started to allow it for free. I forget the name of the back then still competitor.. it worked on me, I retuned to GH back then..
IcyWindows
Do we have a breakdown on how much traffic is free users vs paid users?
reassess_blind
I’d be surprised if paid traffic was more than 1%.
sleepybrett
they could rate limit pushes to free projects.
crote
Absolutely not. This is 100% self-inflicted.
What we are seeing here is a trillion-dollar company screwing over its existing well-established customer base in a desperate attempt to get some crumbs of the current hype cycle. GitHub isn't "dealing with" scaling issues, they are intentionally causing it by shoveling giant piles of coal onto the AI fire.
GitHub's issues would be over tomorrow if they were to put reasonable rate limits on their services. Stop the bots and GitHub is no longer permanently overloaded. It is an intentional business decision not to do this, and instead welcome bots with open arms - to the detriment of their existing paying customers.
I do have some sympathy for the low-level GitHub staff who have to deal with this - especially those who aren't being compensated with insane Bay Area salaries.
mronetwo
We have a business relationship so no we shouldn’t have any sympathy. They sell a service and they’re failing to provide it.
amelius
Why? If we have sympathy for Apple then we can have sympathy for Microsoft ...
mirashii
You're the only mention of Apple in this thread, and I don't see what they have to do with it.
pimeys
Wait, I don't sympathize Apple at all... Or any other American corporation.
itemize123
One, logically, your statement doesn't follow. They're two different companies that have primarily in different market segment.
Two, it's not true in the first place - did we side with apple and its cut of the app store revenue for instance?
jraph
They did that to themselves, and doubly so.
They decided they needed to capture the whole open source ecosystem by turning open source work into social networking... on a proprietary platform (because open source is great, especially when it's others' software). That was before they joined Microsoft.
And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
I would have liked if they had left the open source projects alone and didn't create that FOMO for not using them.
I have no sympathy.
kstrauser
> And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
Ding ding ding, we have a winner. I like AI. I work for an AI company. Still, Microsoft aggressively pushed GitHub users toward Copilot. They don't get to do that and complain about increased volume from AI-generated changes.
No Copilot + reasonable operation: the way things were
Copilot + reasonable operation: Well done!
No Copilot + being overwhelmed by AI commits: Sympathy.
Copilot + being overwhelmed by AI commits: "Where did that petard come from that's hoisting us?"
padjo
I don't typically have sympathy for businesses that fail to deliver a service as advertised.
I can have sympathy for the humans caught in the crossfire but only managing one nine of availability on a commercial service is not acceptable.
bluerooibos
> have a little more sympathy for the billion dollar company
No.
Their leadership went all in on AI and in the last blog post essentially admitted some missing test coverage for a critical path. Time to learn lessons and fix your vibe coded shitslop and stop using "user graph go up" as some kind of excuse.
0xbadcafebee
Our company pays for GitHub. We're paying for a broken product that stops our work. I don't have sympathy for the companies whom I pay for a product and give me broken shit in return. This is entirely preventable and their own fault.
A restaurant makes pizzas. They suddenly get 100x more popular. They can't make 100x more pizzas. But they are still taking orders from 100x more people. Not only are they not getting enough pizzas delivered that they took orders for, but in their rush to make and deliver more pizzas, they set the kitchen on fire, which makes an even longer wait for pizzas.
When the pizza you ordered doesn't get delivered, do you have sympathy for the restaurant? Or do you tell them to stop taking orders they can't fill and try not to set the kitchen on fire?
Now consider the pizza restaurant has 21 billion dollars in cash, is taking your money, and not giving you pizza.
herpdyderp
> I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers.
I don't think this is commendable at all. I give GitHub a lot of money and I'm tired of it being wasted with downtime.
jjice
It's a shame that the GitHub org that we use at my job that we pay a lot of money for gets affected the same way my personal nonsense does.
I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enterprise.
I get the massive load changes that they are under over the last two years, but why does a bunch of vibe coded slop take down the same resources that my company pays for every single month and has for years? I imagine properly splitting that out would be an absolute headache and not worthwhile for them vs stabilizing the rest of the service, but damn it sucks when I get blocked at work because GH is down.
tempest_
Our GitlabCE instance has been sitting in the racks for nearly two years with almost 100% uptime running on 10 year old xeons that have long since paid for themselves.
Of course it is not free of all management but for our use case it is working.
There are hiccups with the CI runners from time time but nothing major and we have another machine in another rack that serves as a backup which can be brought up ~< 20 minutes.
I know companies have long since tossed their expertise for hosting their own stuff in favour of SaaS but at some point its hard to beat the up time of a single machine.
Uvix
We have the expertise. We don’t have the interest in fighting our internal “security” teams to keep running VMs.
snovv_crash
Potato potato.
Expertise isn't just "Do we have a sysadmin?", it's also "Does the org enable them to get their job done?".
a012
You are missing the compliance. Enterprise is mostly compliance, making job done is just second
vel0city
You can pay for GitHub Enterprise Cloud which is essentially their private GitHub deployment stack that they'll host for you. It's theoretically isolated, supposed to be completely separate from these outages.
It's a lot more expensive and has a good bit more limitations to the regular SaaS product.
herpdyderp
As a subscriber to GitHub Enterprise Cloud I assure you that it is not isolated from outages, not in any meaningful way that I have ever noticed.
VCFundedGenYer
"Leave the big trillion dollar corporation alone" is not the right idea.
madrox
I have a lot of sympathy for the people working at GitHub trying to keep it going, but I do not have sympathy for GitHub. That's what they're paid for. I have things to get done, and it has no sympathy for me.
dpz
Maybe for a free account.
But we pay enterprise license and GitHub is a big dependency in our software flow.
If this continues to be a problem as an enterprise product they need to do something. Otherwise theyre are going to to start losing business
anonymousab
> they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
That is an active choice they are making, over and instead of any reliability for the rest of their users.
It is a problem they are embracing, and actively encouraging, for themselves.
Aeolun
Yeah, all those 1100 commits per second. Like, is this the standard for 'scale of traffic' that we're impressed by these days? And it's not like this snuck up on them. The graph shows a very consistent trend that gave them months and months to respond.
rgbrenner
Microsoft made $134B in net profit last year, and is currently valued at nearly $4T. They have the resources to solve this if they wanted to.
chris_money202
Both Microsoft and GH issue is capacity which is being solved with huge capex in DC investment, you can't create a new DC overnight and its even harder in the current climate where everyone assumes the DC is for AI and not just normal workload compute like what GH needs...
KronisLV
> You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
Does nobody use GitLab or Bitbucket or whatever? Why aren't those full of high profile outages as much?
> I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers.
I'd kind of expect that for all non-paid members will get rate limited in some ways, sooner or later. Same as what happened with DockerHub, they're probably just trying to hold out due to the reputational hit such a change would carry.
rpearl
If you do the math, their numbers are something like 1000 commits per second. That's not pushes per second either, it's just the total number of commits summed up over a week, not all of which trigger much of anything to happen.
Yes, that is scale. And yes, that's not actual requests per second. But it's the sort of scale that big (and even mid-sized) tech has known how to deal with for decades. Microsoft doesn't have an excuse.
vrosas
Their excuse is called Azure and if you’ve ever had to deal with it you’d know it’s an amazing(ly awful) excuse to have.
DANmode
Free GitHub, let the engineers choose their own infra.
Bit of a Clownstrike this has become!
JDups
Constant building with AI is something that they (Microsoft) promote and are heavily invested in.
chris_money202
And then get penalized for in the market when they say we need to build more DCs and spend huge capex... which is what GH needs right now
niltecedu
idk man, we pay stupid amounts of money to microsoft, we are an enterprise customer, expecting better availablity compared to my laptop isnt really a high bar.
anon7000
I appreciate it’s tough for the SREs, but GH as a business assumes they must just fall over and keep scaling.
What if they figured out what their platform can actually handle, provided the contractual level of service to their paying customers first, and put some god damn limits on usage.
Like they should just stop signups if they can’t handle the capacity yet. It’s a business decision, not a technical decision, to decide to screw over customers instead of adding limits based on your actual capacity.
flohofwoe
> they're not limiting access to the site or (intentionally) throttling newcomers
They can simply throttle access for everyone to a push/merge frequency which was normal for human-maintained projects.
They should look at how MMOs battle cheating/botting and apply similar countermeasures (it's actually quite absurd that MMOs which are actively attacked every minute have a better uptimes than Github, and on Github the problems are even self-inflicted, not caused by external attacks).
But of course throttling bots to human-level push/merge frequency wouldn't be compatible with the Github leadership's 'AI bullishness'.
alwa
For that matter, I admire their relative transparency about their incidents. I can think of other big players who will cheerfully show green statuses across the board while everyone can see that their pants are down...
stevejb
I don't know enough about Github or Microsoft to say anything intelligent, but I agree with your sentiment in general. And, one thing is for sure - I have used Github for years, pushing many thousands of commits on personal, private projects. I've burned through as many free action minutes as I can, and I use my own action runners as well. I've done this all for free, so on balance I am grateful for them.
bmitc
GitHub literally went all in on doing almost nothing but AI-related feature additions for Copilot. So they built their own prison.
I don't have sympathy for them. They have disrupted my work so much in the past month that it's ridiculous, and my company likely pays them millions of dollars. And you can see from this site's plotting that there is a trend towards more frequent and more critical outages. It has been extremely bad the past month.
whateveracct
isn't github scaling pretty embarrassingly parallel tho? doesn't "ready to scale" at any "good" tech company mean "we can handle 10x without architectural changes?"
that's the mentality i was taught at faang
olivewong
also this site and the news's measure of github's single 9 availability (1 service at all down even copilot) is a bit harsh https://mrshu.github.io/github-statuses/
overfeed
> ...they're dealing with now that everyone is constantly building and pushing with AI.
Also known as the AI chickens coming home to roost. Microsoft was the first to use code uploaded to GitHub to train AI models.
adithyassekhar
I’ve noticed there is 5000 api call limit on gh cli. Haven’t looked at whether it was already there or when does it reset etc. good for them finally if they weren’t already doing that.
mcmcmc
It’s commendable that they let people give them training data for free?
chanux
They probably have bad PR because of huge AI push, Sizeable layoffs etc.
It's probably not fair to the people who are left but alas, this is apparently how the world works.
bamboozled
No, we don’t.
fragmede
The question is, do we think Google would have the same problems running the service? (I'm biased from having worked there tho.)
chanux
I realized they killed Google Cloud Source Repositories and introduced some new product in true Google fashion.
However, YouTube did well where Google Videos did not. Who knows what it would be like if big G bought GH.
ChrisMarshallNY
I strongly suspect that quite a few HN members are busy creating agents that are furiously stuffing GH.
As I said before (and it was not a popular comment), it's easy to be an "armchair quarterback," with these services, as I think the brittleness was already baked in, and just waiting for the right time to crack. The only true way to have a robust platform, is to design something that will scale, from the start, and many startups don't do that, because they are feverishly trying to get out an MVP; even if it is a mess of bubblegum and baling wire.
They always say "We'll get it done right, once we get funding," but that never happens.
But my sympathy is limited by the fact that MS paid a lot of money for this, quite a while ago, and that is one company that knows all about issues of scale. They should have seen this coming.
minaguib
Paying enterprise clients who can not push/pull code and collaborate on PRs (table-stakes) disagree with having a little more sympathy.
theideaofcoffee
Give me a break. Sympathy? For microsoft? That might have flown when github was like seven people, but they have nearly unlimited resources to make it better. They're just choosing not to. Let's talk contracts and money before we pull the sympathy card.
I used to be on-call in a high-traffic environment where single customers pushed more bits than entire nations. I chose the role. I didn't want people's sympathy, if anything, I wanted them to complain to management.
If it gets too bad they can quit. Maybe that would be for the best, just wear the thing down until it outright fails and no one wants to touch it. One less bullshit service sucking all of the oxygen out.
noir_lord
> You could justify the jabs when we could all blame any outage on the migration to Azure, but then they shared numbers around the scale they're dealing with now that everyone is constantly building and pushing with AI.
How about no? They are owned by a company that is neck deep in foisting AI into everything, AI that a lot of us don't want in everything, they are reaping what they seem determined to sow.
So it goes.
BigTTYGothGF
> I think we need to have a little more sympathy for GitHub
We certainly do not.
aduwah
Depends who you are aiming sympathy at. C-levels or Engineering?
evilfred
lol they are Microsoft, and the AI stuff has only multiplied usage like 2x, they haven't had sudden exponential growth
gogobio
Sympathy? It's a Microsoft company that is being ran with a consistency of a startup in early seed rounds. Their downtime is abhorrent and unacceptable as far as enterprise goes. Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
tyre
This is uncharitable, rude, and pretty baseless. Unless you have a lot of direct, personal information about GitHub engineers, they’re dealing with a huge spike in traffic.
Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
atherton94027
Please, read between the lines. Github actions has been a mess since 2019 at least. None of the instability is new, before this unprecedented growth (for a service that's supposed to scale horizontally) the excuse du jour was the azure migration, before that it was the high rate of shipping post acquisition.
Core parts of the product, like navigating to individual files in a code review, are broken
misterderpie
> Core parts of the product, like navigating to individual files in a code review, are broken
I think this is a good argument to underline "It's not _just_ the scale".
Adding to this, the Github Code Review experience is kind-of broken, the way comments/threads are stacked in the PR overview has not improved, pagination isn't really a thing, and these issues are age old.
Hopefully, one day, Github will mature.
atherton94027
It makes you wonder if people review code there given how broken the experience is, it's a core part of the product!
hansvm
The phrasing could have been better, but we've talked as a team about cancelling a $100k/yr contract nearly half a dozen times the last year, and the only thing keeping us from doing so is various "compliance" issues, and to a lesser degree the friction from physically moving. That's a tenuous moat, and if I were a GH PM I'd probably want to know that the current instability is somewhere near critical mass.
linkregister
For CI there are a number of drop-in commercially-available options. You can make it a staged migration, first trivially migrating the build runners, followed by the more complicated integration test and deployment runners. The CI harness itself can follow, and finally moving to a code hosting and visualization service is last.
The final step is challenging; likely the most difficult part is changing all the code references and imports. Shadowing changes would be straightforward. Training your likely 25-50 engineers to use the new code review UX would likely not take that long.
Considering the wasted engineering velocity during Github outages, it's worthwhile to do even a partial migration. Github's action runners have in my experience, been the most fragile part of the platform. Given the ease of moving build and merge queue runners to alternates, it's a no-brainer.
firesteelrain
You are certainly free to attempt to migrate your workloads to your own or leased data center space in your area then buy your own on prem cloud like Dell Private Cloud and deal with all of the maintenance yourself. I don’t think it would amount to $100k/yr and you would be taking on all the risk.
selfhoster1312
For most teams selfhosting on 1U is more than enough which would cost about 1200€/year for housing + let's say 1800€/year for hardware acquisition/depreciation (probably more like 300€/year if you can work with second hand hardware) so that's about 3000€/year or 97% reduction.
Of course you may not want to manage the hardware yourself in which case you'd go for a dedicated server from a reputable provider (i.e. not microsoft/google) like OVH's RISE-XL, Hetzner's EX131, Scaleway's Core-9-L (or an equivalent solution from a smaller provider) for about 4000€/year.
Now let's say you add 8k€/year donations to your distro of choice, forge of choice, and other FLOSS projects and 8k€/year for backups. You got the whole thing running sustainably with >99.9% uptime for 20k€/year or 20% of the original bill.
firesteelrain
You are forgetting the labor of maintaining it to include backups and offsite backups.
jeltz
It would be way less and risk would be lower (It is hard to reach one 9 no matter how bad you are at it), and self hosting is not the only option.
dcow
Self-hosted is not the only alternative...
logicchains
How low would uptime have to get before you'd consider holding the engineers personally responsible for it?
pixelatedindex
Why are we holding the engineers accountable and not the CEO? The company isn’t one department. Responsibility should bubble up to the top.
pydry
They dont have a CEO ever since their last CEO said that human programming "wasn't going anywhere".
pixelatedindex
Is human programming going anywhere? I mean sure you’re not literally typing the actual lines of code, but we haven’t done assembly since compilers and transpilers exist.
Personally, all AI has done is add another layer of abstraction where “pseudo code” can be the source input and out comes the compiled program. You still need to think about the design and maintenance.
vrosas
Agreed. At some point to have to stop hugging your ops team and start firing them.
ksenzee
No, you hire more of them, you give them what they need to do their jobs, and you actually listen to their recommendations. It's not even complicated. It just takes time and money, neither of which companies want to spend on reliability until their customers scream very loud en masse.
Freedom2
Agreed. In fact, keep firing ops until someone gets it done. There's no way your company will build a reputation for not supporting their teams.
mejutoco
Is it a personal attack if it is not directed to a person?
DANmode
Why do I feel like GitHub, without the Microsoft tax, would handle this huge spike in traffic fine?
Can you comment on that?
itemize123
not constructive, but also not personal. rude but based on observed facts.
__MatrixMan__
Sympathy negated by their 2026 Pwnie award: "Lamest Vendor"
Typically when a company gets acquired, they get thrown a bunch of terrible initiatives like “change your cloud provider” that distract from their mission, executives saying “more AI!” and they definitely don’t hear anyone in management say “prioritize technical debt”.
zvmaz
> Their engineers look like absolute amateurs allowing for such low class work it results in their customers experiencing industry leading downtime.
I gather that you have intimate and deep knowledge on the teams and the problems they try to solve there.
californical
If I own a restaurant and buy bread from a supplier — BreadHub.
And 99% of the bread that I get is good but 1% of the loaves, they forgot to add flour. Consistently, for years, they always have loaves missing a key ingredient that I still end up paying for.
I can be pretty sure that BreadHub have a pretty major internal issue, and should probably be questioning their competence, regardless of their “scale”, and without any knowledge of the “problems they’re solving”
tyre
I’m not saying users don’t have a right to be pissed or aren’t justified in looking at other options.
I’m saying that GH is operating at a huge scale with (probably) lots of technical debt and a forced migration to new infrastructure.
I would be (and am) highly critical of leadership. I’m not going to make strong assertions about ICs without knowing their context. I’ve worked at a company with a sterling reputation for engineering excellence where brilliant ICs were kneecapped by poor leadership.
I think a lot of us, at one point or another in our careers, have worked with potato leadership that can be short-sighted or political. It isn’t a comment on the engineers.
selfhoster1312
I mean, the leadership brought it on themselves. For a few years they encouraged sloperating Copilot. Before that, they were encouraging Github Actions usage.
Now the scale problems they're facing is because of those 2 actions of theirs. They didn't have to encourage such growth to turn a profit, and they still don't have to provide gratis services to slopware bots providing 0 value for society.
tryauuum
Please don't do it. Now somebody will come and will try to improve your bread analogy. We will be discussing bread for eons
CoastalCoder
Okay, let's use a car analogy...
collingreen
So a monad is a lot like a burrito...
kayfox
I see you haven't experienced USFoods and Sysco.
Freedom2
I don't think this is an apt analogy. The bread makeup is still there, as far as I'm aware, no users have lost any data or are missing "key" ingredients.
An outage is more like a shipping issue with the supplier, if it's owned wholly by them.
shakna
Okay, let's go with shipping. Current global stat for containers going missing? About 0.0005%.
If any single company reached 1%, they would instantly be out of business, because that is not remotely acceptable, at a global scale.
selfhoster1312
Are you saying the dough is fine but 1% of the batch wasn't cooked? Or rather, that 1% of batches isn't cooked, taking your entire downstream activity down until a new batch arrives?
majormajor
Their engineers almost certainly are not the ones making the decision to let all the new traffic degrade their service for their existing paid customers.
cr125rider
That seems a bit crass but the underlying sentiment stands. Microsoft has more money than God. 4T valuation. When you say you’re worth that much, no excuses. Figure it out.
chris_money202
So your proposal is for Microsoft to invest their free cash in helping support a bunch of AI coders maintain their pet projects? Sounds like you should be a CEO!
robcohen
Start charging more, cut down on CI, figure it out. You don't NEED to give away so much if it means your service is going down.
chris_money202
There isn’t mission critical work being done on GH. You start charging more and the people in charge of finance at companies paying for GH will actually start paying attention. Those finance people don’t care rn because GH is affordable and helps get stuff done. There’s a threshold that exist at every company and once it’s crossed the people who can make money decisions start asking questions
queenkjuul
There is absolutely mission critical work being done on GitHub... There are enterprises paying them for their CI and deployment pipelines
flohofwoe
You don't need to be a CEO to come to the conclusion that throttling to discourage excessive usage is probably a good idea.
chris_money202
It’s not just free users with excessive commits, actions, etc…
Enterprises are benefitting from the same AI and are increasing their volumes as well
queenkjuul
You need to be a CEO to come to the opposite conclusion, apparently
DANmode
The move would have been to avoid purchasing and operating a subsidiary wit a question mark for a budget,
They could throttle git operations to a level which isn't noticeable by normal 'human' usage but seriously slows down automated usage. These are basically DDOS attacks.
javier2
and it has been so damn slow for a number of years now, it drives me insane.
loloquwowndueo
I’d have sympathy for the pre-Microsoft GitHub. For the thing they are now, backed by a top-5 company with trillions of dollars? Cry me a river.
xyst
"sympathy" for one of the worst corporations in history is some insane corp bootlicking logic.
This is a multi billion dollar corporation. History of robbing and stealing from others of their labor or IP. History of enshittifying once great services.
Fuzzwah
Near the end of the 8.5 years that I worked at GitHub as an enterprise support engineer, I asked in an all hands if a "GitHub Classic" product had been considered. Much like World of Warcraft Classic, I imagined it would be a rewrite focused on matching the simpler feature set of the past.
I was basically given the same response that blizzard gave that question; "you think you want that but you don't".
kigiri
Fun idea, I mean, I use old.reddit.com, would probably use classic.github.com too...
Imagine YouTube Classic...
luisfmh
Maybe off topic, but I'm very sad old.reddit.com is now forcing login. Have pretty much stopped using reddit since they did that.
stephenway
nice old.reddit.com, but I don’t think people want something as far back as 2015, they just want their forge to stay boring and fast
ultrarunner
Frankly, even if it was limited only to videos contemporary to whichever point we decided is "classic", I'd still be in.
joshuahedlund
Given that the outages are caused by record traffic, so far they are only cooked in the Yogi Berra sense of “Nobody goes there anymore, it’s too crowded”
Waterluvian
Gets me wondering… if I get used to the fact that some location is busy, I stop going there. Other people might do the same. Could this create a detectable wavy signal over time? People stop going because it’s busy. It stops being busy. People go because it’s not busy. Repeat. Is this a known phenomenon?
Random AI slop code pushers don't pay the bills. Traffic isn't revenue.
I've started suggesting to my CTO that we move to something else because we do pay the Github bills and yet we are also affected by other people's activity. Avoiding that is not a hard engineering problem for an alternative.
A world where a record amount of code is pushed to GH but a large amount of paying customers leave would be "cooked"
olyjohn
Yeah I don't feel bad for Microsoft, but I don't feel bad for people who keep paying them for it either. This has been going on forever now. Nothing will happen until people start leaving. I don't honestly understand why so much shit is on GitHub when there are so many alternatives, anyways. It's fucking git! Can't people do or try anything different, ever?
pocksuppet
People are on GitHub because it's the cool thing to do, and there is no concept of anything else that could matter.
angry_octet
It is quite possible for github load balancers to route paying customers to different queues/infrastructure. I think much of the bot activity is paying customers, they just don't have per-client QoS because that would be hard. And they don't have separate auth for bots so they can't queue them separately.
Bratmon
Many people think we're on the path to having separate webs for AIs and humans. This may be the beginning of that.
bluefirebrand
As much as I would like that to happen, I don't think it is at all possible until we solve the "is a human" problem.
JeremyHerrman
> "GitHub has had 1125 incidents since February 2016, implying a monthly incident rate of 24"
1125 incidents / 126 months ≈ 8.9 incidents per month, not 24
still terrible, but why such an obvious error in the first sentence...
graypegg
Ahhh, I think the author mixed up two values here. That value seems to actually be the average over the past 3 months.
const incidents = e.detail.incidents;
// ...snip...
const now = new Date();
const threeMonthsAgo = new Date(now);
threeMonthsAgo.setMonth(threeMonthsAgo.getMonth() - 3);
// ...snip...
var currentFreq = recentIncidents.length / 3; // <- We out here, smoking these guns with our homeboy Claude
// ...snip...
var earliest = null;
for (var j = 0; j < incidents.length; j++) {
var dd = new Date(incidents[j].started_at); // <- eventually incidents[j].started_at is "2016-03-01T07:07:37.000Z"
if (!earliest || dd < earliest) earliest = dd;
}
// ...snip...
document.getElementById('n-since').textContent = earliest
? earliest.toLocaleDateString('en-US', { month: 'long', year: 'numeric' })
: '?';
document.getElementById('n-rate').textContent = Math.round(currentFreq * 10) / 10;
#n-since is going to be either march or feburary. It'll change depending on your timezone because JS's Date object always shifts the date around to match the same instant but in the system's timezone.
#n-rate has nothing to do with the #n-since month, it's just the last trailing 3 months. And even then, it's sort of underbaked? It's moving the date back by 3 calendar months not taking into account differing numbers of days, so it'll under-report short months.
I wouldn't trust the stats here.
Edit: whoops, author updated the template while I was writing this! It now says "Over the last 3 months", though that's still calendar months.
6LLvveMx2koXfwn
Not sure whether it has been updated since your comment, but the sentence now reads:
GitHub has had 1125 incidents since March 2016. Over the last 3 months, they've averaged 24 incidents per month
edit: although they also have 1.2 days of downtime (in a day) for their 'worst days' of downtime table, which suggests some auto number crunching is not working as expected.
gen220
Yes I tweaked it! The number and copy were mismatched and are no longer!
That worst day is likely an overlapping incidents accounting issue; I tried to account for overlapping incidents in another view but probably failed to port it over there.
Should be fixed soon!
stevage
> GitHub has had 1125 incidents since March 2016. Over the last 3 months, they've averaged 24 incidents per month (↓ 5% vs prev 3mo).
Looks like they fixed it already
4petesake
Prob used Copilot to write the excel formula...
_heimdall
I feel for the team having to deal with these problems at github today. Its also beyond me how those in charge for years wouldn't have seen this coming.
Github was bought by Microsoft, whether they wanted to acknowledge that internally or not. Microsoft went deep on LLMs, and specifically on LLMs for coding use cases. They must have recognized that LLM generated code and PRs would effectively DDoS GitHub.
I can only assume they simply didn't care, likely driven by greed.
404mm
If backend GitHub services are anything like GHES then I’m surprised it even managed to scale this much.
angry_octet
I think its nothing like the Enterprise version, which is why they hate having to maintain it, same with Atlassian and Azure/Entra AD. Once service to rule them all, and in the cloud bind them.
For security reasons, I would love a low feature, carefully engineered, offline github-like system. Something tuned for agentic behaviours, with an understanding of hierarchical agent identities.
xyzzy_plugh
Is GitHub Cooked? Decidedly yes.
Migrating to something else has been raised as a concern during every engagement with clients and prospective clients in the past year.
"Do you use GitHub?"
"Yes though we'd like to move to something else, we just don't know what yet."
As soon as the next big thing shows up, they're done. And they know it.
Bratmon
I wonder if this is a Microsoft Excel situation where everyone
1. Hates the status quo
2. Only uses a handful of the
10,000 features the status quo has
3. Uses a different handful than everyone else
Because that model kept Excel as the center of buisnesses around the world for 30 years and counting
jeremyjh
People don't hate Excel. Well - maybe programmers who don't use it hate it.
In reality, Excel is the only reason Microsoft still exists.
Excel is how they lure you in. Excel really is the best spreadsheet, its the only one every doc you are sent works with, but also the one people know inside and out. And if you are into Pivot tables, PowerQuery, etc there is just no escaping it. You are utterly stuck.
But here is how they get you: its not much more to rent the whole office suite compared to just Excel. And as long as you are into it that far, Outlook and Teams are much cheaper than GSuite + Slack. And still...okish? No, not really. But we pretend. And it wouldn't happen without Excel.
fulafel
Microsoft has many products that people hate but are still the reason why they exist (eg Windows, Teams, Sharepoint etc).
djieidj283
It’s ironic that the SCM that 2026 software engineers landed on, is one with a complicated distributed usage model and is slow/down because of a centralised service
chanux
Apparently it's not exactly git side of GitHub that's failing. It's all the bells and whistles bolted on later causing the trouble.
kevmo
An important thing to consider is how much of their uptime without incidents is not the normal working hours. Their incident-free uptime on 9-5 EST, Mon-Fri, is probably like 60%.
thombles
As a daily GitHub user in Australia I still haven’t figured out why everyone’s complaining about uptime. :)
CoastalCoder
> Their incident-free uptime on 9-5 EST, Mon-Fri, is probably like 60%.
And it may be even worse in EDT, which is currently in effect!
perfectstorm
what's normal working hours? very US centric comment IMO. Europe, India, China, Latam etc. don't fall into your 9-5 EST normal working hour bucket.
padjo
A service like GH will still show a daily usage pattern, often with a peak somewhere around 16:00 UTC when most of the US and Europe are at work.
boredatoms
Im fairly certain that SWEs only exist on the US west coast
jen20
They would be in PST then, no?
nightpool
Getting rid of Actions and Copilot and other secondary services almost halves Github's incident rate: https://i.imgur.com/XPcMIFr.png
I'm a big fan of Github Actions and I think people are often a little too harsh on it, but it's clear that it's sad that it's come at such a high cost to the platform's stability
Normal_gaussian
Actions aren't secondary to most paid GH users; and if they are down it usually means no deployments and no tests, which can often halt work.
isityettime
GitHub Actions is really, really badly designed. The security model is fundamentally broken, the YAML hell is as bad as any, the log streaming lags like hell, they charge self-hosted runners for using their coordination plane, jobs queuing is really slow, and their software for actually running jobs is cursed and designed in a way that is practically hostile to self-hosting.
GitHub Actions is god-awful. Have you ever used any other CI tools?
nightpool
Yes, I was actually a part-time Jenkins admin back in high school and college. I use CircleCI more than Actions most of the time (and used Travis before that), especially for higher cost workflows, but there's no beating Actions in terms of its ease of setup and ecosystem advantage. It's way easier to just set up a simple JS file that runs on every commit in Actions compared to anywhere else.
Would be happy to be proven wrong!
electroly
> they charge self-hosted runners for using their coordination plane
They don't. Self-hosted runners are still free. They backed down from their plan to charge after receiving widespread backlash.
imnes
It's not making headlines, but Google and Apple seem to be getting hammered by AI as well. Getting app updates approved to release on prod on Play Store / App store used to take days, now it's taking weeks.
silver92bullet
I hear the comments that state we should have sympathy for github. For the individual ops/sre I can have that sympathy because it must a heck of a time internally to manage things. I can't however have sympathy for company itself. They have not set themselves or the community for success. I think it is the responsibility of the company to anticipate the issues or at least react in a way that would build trust. The fact is there is a systemic issue in the company that causes the repeated unreliableness and they have not rooted it out.
crossroadsguy
But after exploring some options for small but private and personal repos (not foss; not yet at least) I realised I am better off with a free Github private repo. There simply isn't a feasible free alternative for such a use-case without hitting, often a 50MB to say few 100s MB, the limit ASAP. There's gitlab.com, bitbucket (? haven't opened it in ages) but those are not better in any way for use cases like mine.
Are there any?
arccy
> free
you are the problem
crossroadsguy
> > free
> you are the problem
Hey, arccy.
What a pathetic one-liner reply!
(update: since it was being downvoted anyway, removed the explanations part and just left the pissed off longer knee jerk reaction response to another shorter knee jerk reaction)
Coming back to your pathetic quip: what kind of piece of work one has to be to reply in this somewhat apple-fanboi-esque way (where someone says "how to fix X problem in Y" and pat comes the reply "why not just buy Y-latest, it's just $349") and what kind of unidimensional world you live in where you believe someone wanting some free and private git hosting is the problem? Ffs.
So again and with all due respect, please shut the fuck up! Because you don't know me, you don't know where I am coming from, you don't know the why, and you don't come across as someone who would ask or even care about knowing.
owebmaster
His one-liner was terrible but it made you waste so much of your time getting angry, which is what feed the trolls.
SaudAljuaid
The enterprise vs free‑tier tension is real. Scaling for unpredictable AI traffic must be rough. Interesting to read everyone’s perspectives here.
devy
Didn't GitHub CTO Vladimir Fedorov just posted this a couple days ago?
github.com expires in 44 days
github.com's SSL certificate expires in 34 days.
Will either cause an outage next month?
tdg5
I’ve reached the point where I just feel a mourning kind of sadness for GitHub
flyingshelf
I feel that way for open source and programming in general. I don’t think it's going to be a thing anymore in just a couple of years. Even now I see people already don't look at any of the code they release.
microcannot
What most people don't understand is that Microsoft can no longer engineer a technology product, they simply can't develop anything anymore. They can throw slop around, but they lack the corporate culture to actually build an innovative technology product. They can purchase other companies that have built products, but as soon as they do those products stop innovating and begin dying.
kuuuzya
Oh, nice, but whats the difference with their own tool?
GrumpySciGuy
Yes, but I did not need to look at the tracker to know that.....
If you just need a place to host your stuff, GitHub may be good enough. But for an organization that requires a workflow it is utterly inadequate.
MS is trying to catch two hares at the same time, which has never worked: Slop hosting for the masses and workflow provider for the relatively few. You can't have both. But MS won't make the hard decision. Those who need workflow are on their own to find alternatives.
microcantanymor
What most people don't understand is that Microsoft can no longer engineer a technology product, they simply can't develop anything anymore. They can throw slop around, but they lack the corporate culture to actually build an innovative technology product. They can purchase other companies that have built products, but as soon as they do those products stop innovating and begin dying.
sitzkrieg
using github in 2026 is a self own. its that simple at this point
arlattimore
I literally chuckled out loud when I saw the headline :D
Khaine
Maybe they should bring back the meritocracy rug
xyst
Now add an rss/atom feed link and you have my support
Hey! isgithubcooked.com is my site; I made this site in February '26, I think
The contribution graph as an outage calendar idea is a commonly recurring one :). I definitely saw it somewhere else as a static asset first before I made this site.
6 years is a good run. For comparison, every 6 years there's a not insignificant chance you die. A restaurant that lasted 6 years would be an outstanding success. 6 years ago the very first LLMs were a novelty toy. 6 years before that was just after HTTPS became popular. 6 years before that was about when the iPhone came out.
I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers. Yes, they need to get this figured out, but a little sympathy goes a long way. I personally wish them the best and hope their on-call people can go back to getting normal amounts of sleep soon.
1. Github has enterprise users who paid for the service, their day job requires Github to be available and working
2. Github has generous free tier which is the one which is exploring a lot more with the AI generated code.
It is a complexity in itself but the traffic should have been separated, the free users should not be allowed to bring down Github for enterprise customers (Just to clarify, I am free user myself). And if they do not have capacity it would have been perfectly fine to push back or throttle new users/repositories.
But I think they’re in a tough spot. GitHub has historically been a huge supporter, proponent, and provider for open source projects. Engineers are difficult customers, to say the least, and the community would likely freak tf out of the segmented traffic.
The logical, pragmatic, and justifiable answer doesn’t always align with your market.
They are in a tough spot cause they want to continuously ingest all the data to train their LLMs. Any fork or decentralization at large scale of git is going to impact the training pipeline.
What we are seeing here is a trillion-dollar company screwing over its existing well-established customer base in a desperate attempt to get some crumbs of the current hype cycle. GitHub isn't "dealing with" scaling issues, they are intentionally causing it by shoveling giant piles of coal onto the AI fire.
GitHub's issues would be over tomorrow if they were to put reasonable rate limits on their services. Stop the bots and GitHub is no longer permanently overloaded. It is an intentional business decision not to do this, and instead welcome bots with open arms - to the detriment of their existing paying customers.
I do have some sympathy for the low-level GitHub staff who have to deal with this - especially those who aren't being compensated with insane Bay Area salaries.
They decided they needed to capture the whole open source ecosystem by turning open source work into social networking... on a proprietary platform (because open source is great, especially when it's others' software). That was before they joined Microsoft.
And then Microsoft pushed AI everywhere, including on GitHub itself with copilot.
I would have liked if they had left the open source projects alone and didn't create that FOMO for not using them.
I have no sympathy.
Ding ding ding, we have a winner. I like AI. I work for an AI company. Still, Microsoft aggressively pushed GitHub users toward Copilot. They don't get to do that and complain about increased volume from AI-generated changes.
No Copilot + reasonable operation: the way things were
Copilot + reasonable operation: Well done!
No Copilot + being overwhelmed by AI commits: Sympathy.
Copilot + being overwhelmed by AI commits: "Where did that petard come from that's hoisting us?"
I can have sympathy for the humans caught in the crossfire but only managing one nine of availability on a commercial service is not acceptable.
No.
Their leadership went all in on AI and in the last blog post essentially admitted some missing test coverage for a critical path. Time to learn lessons and fix your vibe coded shitslop and stop using "user graph go up" as some kind of excuse.
A restaurant makes pizzas. They suddenly get 100x more popular. They can't make 100x more pizzas. But they are still taking orders from 100x more people. Not only are they not getting enough pizzas delivered that they took orders for, but in their rush to make and deliver more pizzas, they set the kitchen on fire, which makes an even longer wait for pizzas.
When the pizza you ordered doesn't get delivered, do you have sympathy for the restaurant? Or do you tell them to stop taking orders they can't fill and try not to set the kitchen on fire?
Now consider the pizza restaurant has 21 billion dollars in cash, is taking your money, and not giving you pizza.
I don't think this is commendable at all. I give GitHub a lot of money and I'm tired of it being wasted with downtime.
I don't know the architecture or any of that, but I feel like there could be (and it's not like they would've really known this until the last year or two with the massive spike) separate infrastructure for paid users/orgs vs free the same way they make the distinction with enterprise.
I get the massive load changes that they are under over the last two years, but why does a bunch of vibe coded slop take down the same resources that my company pays for every single month and has for years? I imagine properly splitting that out would be an absolute headache and not worthwhile for them vs stabilizing the rest of the service, but damn it sucks when I get blocked at work because GH is down.
Of course it is not free of all management but for our use case it is working.
There are hiccups with the CI runners from time time but nothing major and we have another machine in another rack that serves as a backup which can be brought up ~< 20 minutes.
I know companies have long since tossed their expertise for hosting their own stuff in favour of SaaS but at some point its hard to beat the up time of a single machine.
Expertise isn't just "Do we have a sysadmin?", it's also "Does the org enable them to get their job done?".
It's a lot more expensive and has a good bit more limitations to the regular SaaS product.
But we pay enterprise license and GitHub is a big dependency in our software flow.
If this continues to be a problem as an enterprise product they need to do something. Otherwise theyre are going to to start losing business
That is an active choice they are making, over and instead of any reliability for the rest of their users.
It is a problem they are embracing, and actively encouraging, for themselves.
Does nobody use GitLab or Bitbucket or whatever? Why aren't those full of high profile outages as much?
> I think it's commendable that they're not limiting access to the site or (intentionally) throttling newcomers.
I'd kind of expect that for all non-paid members will get rate limited in some ways, sooner or later. Same as what happened with DockerHub, they're probably just trying to hold out due to the reputational hit such a change would carry.
Yes, that is scale. And yes, that's not actual requests per second. But it's the sort of scale that big (and even mid-sized) tech has known how to deal with for decades. Microsoft doesn't have an excuse.
Bit of a Clownstrike this has become!
What if they figured out what their platform can actually handle, provided the contractual level of service to their paying customers first, and put some god damn limits on usage.
Like they should just stop signups if they can’t handle the capacity yet. It’s a business decision, not a technical decision, to decide to screw over customers instead of adding limits based on your actual capacity.
They can simply throttle access for everyone to a push/merge frequency which was normal for human-maintained projects.
They should look at how MMOs battle cheating/botting and apply similar countermeasures (it's actually quite absurd that MMOs which are actively attacked every minute have a better uptimes than Github, and on Github the problems are even self-inflicted, not caused by external attacks).
But of course throttling bots to human-level push/merge frequency wouldn't be compatible with the Github leadership's 'AI bullishness'.
I don't have sympathy for them. They have disrupted my work so much in the past month that it's ridiculous, and my company likely pays them millions of dollars. And you can see from this site's plotting that there is a trend towards more frequent and more critical outages. It has been extremely bad the past month.
that's the mentality i was taught at faang
Also known as the AI chickens coming home to roost. Microsoft was the first to use code uploaded to GitHub to train AI models.
It's probably not fair to the people who are left but alas, this is apparently how the world works.
However, YouTube did well where Google Videos did not. Who knows what it would be like if big G bought GH.
As I said before (and it was not a popular comment), it's easy to be an "armchair quarterback," with these services, as I think the brittleness was already baked in, and just waiting for the right time to crack. The only true way to have a robust platform, is to design something that will scale, from the start, and many startups don't do that, because they are feverishly trying to get out an MVP; even if it is a mess of bubblegum and baling wire.
They always say "We'll get it done right, once we get funding," but that never happens.
But my sympathy is limited by the fact that MS paid a lot of money for this, quite a while ago, and that is one company that knows all about issues of scale. They should have seen this coming.
I used to be on-call in a high-traffic environment where single customers pushed more bits than entire nations. I chose the role. I didn't want people's sympathy, if anything, I wanted them to complain to management.
If it gets too bad they can quit. Maybe that would be for the best, just wear the thing down until it outright fails and no one wants to touch it. One less bullshit service sucking all of the oxygen out.
How about no? They are owned by a company that is neck deep in foisting AI into everything, AI that a lot of us don't want in everything, they are reaping what they seem determined to sow.
So it goes.
We certainly do not.
Is their uptime acceptable? No. But personal attacks aren’t necessary or constructive.
Core parts of the product, like navigating to individual files in a code review, are broken
I think this is a good argument to underline "It's not _just_ the scale". Adding to this, the Github Code Review experience is kind-of broken, the way comments/threads are stacked in the PR overview has not improved, pagination isn't really a thing, and these issues are age old. Hopefully, one day, Github will mature.
The final step is challenging; likely the most difficult part is changing all the code references and imports. Shadowing changes would be straightforward. Training your likely 25-50 engineers to use the new code review UX would likely not take that long.
Considering the wasted engineering velocity during Github outages, it's worthwhile to do even a partial migration. Github's action runners have in my experience, been the most fragile part of the platform. Given the ease of moving build and merge queue runners to alternates, it's a no-brainer.
Of course you may not want to manage the hardware yourself in which case you'd go for a dedicated server from a reputable provider (i.e. not microsoft/google) like OVH's RISE-XL, Hetzner's EX131, Scaleway's Core-9-L (or an equivalent solution from a smaller provider) for about 4000€/year.
Now let's say you add 8k€/year donations to your distro of choice, forge of choice, and other FLOSS projects and 8k€/year for backups. You got the whole thing running sustainably with >99.9% uptime for 20k€/year or 20% of the original bill.
Personally, all AI has done is add another layer of abstraction where “pseudo code” can be the source input and out comes the compiled program. You still need to think about the design and maintenance.
Can you comment on that?
https://this.weekinsecurity.com/microsoft-wins-lamest-vendor...
I gather that you have intimate and deep knowledge on the teams and the problems they try to solve there.
And 99% of the bread that I get is good but 1% of the loaves, they forgot to add flour. Consistently, for years, they always have loaves missing a key ingredient that I still end up paying for.
I can be pretty sure that BreadHub have a pretty major internal issue, and should probably be questioning their competence, regardless of their “scale”, and without any knowledge of the “problems they’re solving”
I’m saying that GH is operating at a huge scale with (probably) lots of technical debt and a forced migration to new infrastructure.
I would be (and am) highly critical of leadership. I’m not going to make strong assertions about ICs without knowing their context. I’ve worked at a company with a sterling reputation for engineering excellence where brilliant ICs were kneecapped by poor leadership.
I think a lot of us, at one point or another in our careers, have worked with potato leadership that can be short-sighted or political. It isn’t a comment on the engineers.
Now the scale problems they're facing is because of those 2 actions of theirs. They didn't have to encourage such growth to turn a profit, and they still don't have to provide gratis services to slopware bots providing 0 value for society.
An outage is more like a shipping issue with the supplier, if it's owned wholly by them.
If any single company reached 1%, they would instantly be out of business, because that is not remotely acceptable, at a global scale.
Enterprises are benefitting from the same AI and are increasing their volumes as well
unless the firm was prepared to see it through.
This is a multi billion dollar corporation. History of robbing and stealing from others of their labor or IP. History of enshittifying once great services.
I was basically given the same response that blizzard gave that question; "you think you want that but you don't".
Imagine YouTube Classic...
https://en.wikipedia.org/wiki/El_Farol_Bar_problem
I've started suggesting to my CTO that we move to something else because we do pay the Github bills and yet we are also affected by other people's activity. Avoiding that is not a hard engineering problem for an alternative.
A world where a record amount of code is pushed to GH but a large amount of paying customers leave would be "cooked"
1125 incidents / 126 months ≈ 8.9 incidents per month, not 24
still terrible, but why such an obvious error in the first sentence...
#n-rate has nothing to do with the #n-since month, it's just the last trailing 3 months. And even then, it's sort of underbaked? It's moving the date back by 3 calendar months not taking into account differing numbers of days, so it'll under-report short months.
I wouldn't trust the stats here.
Edit: whoops, author updated the template while I was writing this! It now says "Over the last 3 months", though that's still calendar months.
That worst day is likely an overlapping incidents accounting issue; I tried to account for overlapping incidents in another view but probably failed to port it over there.
Should be fixed soon!
Looks like they fixed it already
Github was bought by Microsoft, whether they wanted to acknowledge that internally or not. Microsoft went deep on LLMs, and specifically on LLMs for coding use cases. They must have recognized that LLM generated code and PRs would effectively DDoS GitHub.
I can only assume they simply didn't care, likely driven by greed.
For security reasons, I would love a low feature, carefully engineered, offline github-like system. Something tuned for agentic behaviours, with an understanding of hierarchical agent identities.
Migrating to something else has been raised as a concern during every engagement with clients and prospective clients in the past year.
"Do you use GitHub?"
"Yes though we'd like to move to something else, we just don't know what yet."
As soon as the next big thing shows up, they're done. And they know it.
1. Hates the status quo
2. Only uses a handful of the 10,000 features the status quo has
3. Uses a different handful than everyone else
Because that model kept Excel as the center of buisnesses around the world for 30 years and counting
In reality, Excel is the only reason Microsoft still exists.
Excel is how they lure you in. Excel really is the best spreadsheet, its the only one every doc you are sent works with, but also the one people know inside and out. And if you are into Pivot tables, PowerQuery, etc there is just no escaping it. You are utterly stuck.
But here is how they get you: its not much more to rent the whole office suite compared to just Excel. And as long as you are into it that far, Outlook and Teams are much cheaper than GSuite + Slack. And still...okish? No, not really. But we pretend. And it wouldn't happen without Excel.
And it may be even worse in EDT, which is currently in effect!
I'm a big fan of Github Actions and I think people are often a little too harsh on it, but it's clear that it's sad that it's come at such a high cost to the platform's stability
GitHub Actions is god-awful. Have you ever used any other CI tools?
Would be happy to be proven wrong!
They don't. Self-hosted runners are still free. They backed down from their plan to charge after receiving widespread backlash.
Are there any?
you are the problem
> you are the problem
Hey, arccy.
What a pathetic one-liner reply!
(update: since it was being downvoted anyway, removed the explanations part and just left the pissed off longer knee jerk reaction response to another shorter knee jerk reaction)
Coming back to your pathetic quip: what kind of piece of work one has to be to reply in this somewhat apple-fanboi-esque way (where someone says "how to fix X problem in Y" and pat comes the reply "why not just buy Y-latest, it's just $349") and what kind of unidimensional world you live in where you believe someone wanting some free and private git hosting is the problem? Ffs.
So again and with all due respect, please shut the fuck up! Because you don't know me, you don't know where I am coming from, you don't know the why, and you don't come across as someone who would ask or even care about knowing.
https://github.blog/news-insights/company-news/the-august-17...
Will either cause an outage next month?
I was ready to click...
MS is trying to catch two hares at the same time, which has never worked: Slop hosting for the masses and workflow provider for the relatively few. You can't have both. But MS won't make the hard decision. Those who need workflow are on their own to find alternatives.
https://red-squares.cian.lol/
The contribution graph as an outage calendar idea is a commonly recurring one :). I definitely saw it somewhere else as a static asset first before I made this site.
Any dev can just have their own git mirror and work there, but replacing github is not really feasable, not one of the other services come even close.
If github is down for say 2h i dont really care. I can live with that. IF i did care i would just host my own.
This is why I foresaw that centralizing everything to GitHub was just generally a bad idea 6 years ago. [0]
Now that there is no CEO of GitHub, there is no point to GitHub improving.
[0] https://news.ycombinator.com/item?id=22867803