Artwork for podcast Tag1 Team Talks | The Tag1 Consulting Podcast
Using the Goose load testing framework to troubleshoot Quick Edit - Tag1 Team Talk
Episode 8229th September 2021 • Tag1 Team Talks | The Tag1 Consulting Podcast • Tag1 Consulting, Inc.
00:00:00 00:32:47

Share Episode

Shownotes

In a previous Tag1 Team Talk, we discussed How to analyze Goose load testing results using New Relic. During testing, a bug was discovered! This team talk goes into detail about what the bug was, how they narrowed down the scope of the problem, and how to fix it. 

In this test, Goose found a problem with Drupal’s Quick Edit feature. Quick Edit is a staple of content creators, enabling them to make changes directly in content, even if that content is wrapped in blocks or other Drupal layout features.

Once again, join Goose creator and CEO of Tag1 Consulting Jeremy Andrews, VP of Software Engineering and Goose contributor Fabian Franz, and Managing Director Michael Meyers as they dig into the results of a Goose Attack, and how this Goose test found something that other analyses did not. 

Transcripts

Speaker:

Hello, and welcome to Tag1 Team Talks, the podcast and blog of Tag1 Consulting.

Speaker:

Today, we're going to be talking more about finding and fixing problems

Speaker:

with Goose, an open source load testing framework that was inspired

Speaker:

by Locust and written in Rust.

Speaker:

It's the most scalable framework available.

Speaker:

It is the easiest to scale.

Speaker:

You can check out all of our talks on Goose at Tag1.com/Goose.

Speaker:

Today's talk is a really great example of why you should you load tests over

Speaker:

long periods of time and not just short runs as well as why you should be

Speaker:

performing edit actions on, you know, creating content and performing editing

Speaker:

actions on the content that you create.

Speaker:

During our last talk on Goose, we explored the practical side of load testing, and

Speaker:

we walked through how to analyze your load testing results, using New Relic

Speaker:

so that you can find problems that are impacting your website and address them.

Speaker:

We load tested against the Drupal framework and CMS with the default

Speaker:

umami install and in the process, we did discover a problem.

Speaker:

Uh, the problem is with Quick Edit, Quick Edit allows you to quickly

Speaker:

edit your content without having to leave the context of the page

Speaker:

and going into full edit mode.

Speaker:

What we found in running a load test over a long period of time and creating

Speaker:

and editing content is that when you have a lot of revisions, load times,

Speaker:

get insanely slow for end-users.

Speaker:

We're going to walk through how we use the local container to track down

Speaker:

the bug, fixed it, and then confirm the fix with follow up load tests.

Speaker:

I'm Michael Meyers, managing director of Tag1 Consulting.

Speaker:

And joining me today are two performance and scalability experts.

Speaker:

Jeremy Andrews, the founder and CEO of Tag1 and the original creator of Goose

Speaker:

and Fabian Franz our VP of technology.

Speaker:

Who's been a major contributor to Goose, uh, and a really well known

Speaker:

performance and scalability expert.

Speaker:

Uh, Fabian is responsible for a lot of the performance and scalability in Drupal.

Speaker:

Jeremy, Fabian, thank you guys so much for joining.

Speaker:

I'm excited that we, uh, found a bug in our, in our load test.

Speaker:

Not that there is a bug, but that, you know, we set out to do a load test and

Speaker:

sure enough, uh, we found one, uh, and I thought, you know, I learned so much in

Speaker:

how to track things down with New Relic.

Speaker:

It's, it's this amazing tool.

Speaker:

Uh, Fabian, could you tell us, you know, uh, just rewind a little bit, tell folks

Speaker:

how we discovered the bug, you know, what jumped out and what the next steps were.

Speaker:

So essentially what we did, um, on New Relic, we, um, tested our beautiful Umami

Speaker:

install and did that for quite awhile.

Speaker:

So we run it for a few hours, both for testing the load test.

Speaker:

Um, so that'd be had good demos and, um, then later just to demonstrate it and

Speaker:

we found that over time it gets slower.

Speaker:

Um, but we also found, um, that we had several slow responses always there.

Speaker:

Things like 27 seconds.

Speaker:

And then we found that all those responses, especially, um, when we

Speaker:

looked at a partial trace and New Relic and we found this one function,

Speaker:

quick edit entity view all the time.

Speaker:

Um, so what apparently quick edit did something here.

Speaker:

So, uh, let me share my screen.

Speaker:

I'm gonna, uh, gonna start here?

Speaker:

this is Umami.

Speaker:

I'm logged in as an admin and just to show real quick what Quick Edit is.

Speaker:

It's so I can just edit this and then save it if I want to,

Speaker:

or I can just start my changes.

Speaker:

That's quick edit,.

Speaker:

in a nutshell.

Speaker:

Um, the bug we found actually is pretty well-known.

Speaker:

Um, it's a regression from 2018, was created 17, March, 2018,

Speaker:

and, uh, it still needs work.

Speaker:

Uh, it's still not fixed in Drupal 9.3.x and it's a critical

Speaker:

part of the entity system.

Speaker:

And here we can already see, some people have already debugged

Speaker:

something, but we are not going to look at that because, when I started

Speaker:

debugging, I didn't have that.

Speaker:

So let's assume you don't.

Speaker:

So, um, the first thing I did was, I ran a Goose load test against my

Speaker:

umami instance, but surprise the problem that was so apparent on

Speaker:

New Relic was just not to be found.

Speaker:

So, um, then I thought maybe it's something that has to do,

Speaker:

that just happens after a while.

Speaker:

And so I did leave a Goose running overnight and, uh, came back the next

Speaker:

morning ,and voila, the problem was there.

Speaker:

Then I looked what has changed essentially.

Speaker:

And what has changed is that whenever within a standard Umami

Speaker:

profile an article is edited.

Speaker:

Um, you create a new revision of it.

Speaker:

So, here we have, several revisions created by admin, uh, which are

Speaker:

created by the load test and one originally as part of this profile.

Speaker:

Okay.

Speaker:

So it had something to do with revisions.

Speaker:

That was a good point.

Speaker:

So, what I then did was, I created, kind of a little, um, a shell script.

Speaker:

there was just, deprecating one revision entry, um, because I wanted

Speaker:

to really analyse the problem turns out 100,000 revisions it's way too much.

Speaker:

We'll later see, what's that about, but essentially in the end I've I

Speaker:

want to have, um, 500, uh, revisions.

Speaker:

So what we can do is we could count the number of revisions from a node revision.

Speaker:

Um, group by node id.

Speaker:

So then, and we see, um, many, you just have little ones, so this is no

Speaker:

longer the installation I let run, Goose overnight, but this is a new installation.

Speaker:

And then just input like around 6,000 revisions, um,

Speaker:

just to -------- the problem.

Speaker:

Um, so, this is how is the revision table looks.And then we can

Speaker:

essentially start with two things.

Speaker:

Uh, we can start from the code level and just look at what's

Speaker:

calling Quick Edit into view alter.

Speaker:

What does set calling?

Speaker:

What is that calling what's that calling and go there, or we can

Speaker:

now that we control our local, uh, essentially get a trace.

Speaker:

Um, and for that we start a normal load test.

Speaker:

Um, just like normal.

Speaker:

So we have our Goose here, admin username, password is already in the environment and

Speaker:

we are just starting it up and, and, um, because I have so many revisions, Goose,

Speaker:

should be able to, show me the problem.

Speaker:

So while were waiting for Goose to do that,

Speaker:

um, I've also put us a little, uh, nice tool, uh, here, um, into our,

Speaker:

installation and, we're using, for this whole installation here we're

Speaker:

using DDev um, that means, again, for example, the DDev Drush and execute

Speaker:

Drush commands on this installation, or I could, quickly get, uh, get a

Speaker:

database connection, via drush sql-cli.

Speaker:

Um, and, um, So DDev is the core installation used.

Speaker:

And, uh, but what we really would like is kind of like a local New Relic so

Speaker:

that we can, um, essentially record all those, um, runs that are doing.

Speaker:

And this is what I did.

Speaker:

I installed a tool called XH graphical user interface.

Speaker:

And, um, it's a little bit old, but still working at users among the DB container.

Speaker:

And it uses, um, a web container to then show me all the recent ones.

Speaker:

And here you can see whenever Goose is visiting a page.

Speaker:

I am seeing here my, um, my traces, but it's really, really nice.

Speaker:

And, uh, despite New Relic, which obviously also needs to

Speaker:

work this production instances.

Speaker:

And we did test on a kind of production instance our load test last time.

Speaker:

Uh, We only get partial traces unless we configure, especially, um, here I

Speaker:

get full traces because I don't care as much about the performance and

Speaker:

more about the relative performance.

Speaker:

So we can already see, do we have something with the longest waittime?

Speaker:

Yes, we have.

Speaker:

So here we have one that took 22 second, 22nd, 19 seconds, 18 seconds.

Speaker:

17 seconds.

Speaker:

Okay.

Speaker:

So, uh, always the same article, funnily enough.

Speaker:

Uh, okay.

Speaker:

So, probably one that gets ended frequently.

Speaker:

Um, we could now, uh, look into one of those ones and see already,

Speaker:

oh, it's something with the database that takes so long, the

Speaker:

remainder is kind of, not really.

Speaker:

And you can see this graphical user interface a lot more raw than New

Speaker:

Relic in that, but we can still get a, get a pretty nice idea.

Speaker:

So, um, there's two ways we can now, essentially get to,

Speaker:

oh there's some nice graphs.

Speaker:

Um, who's using what memory, et cetera.

Speaker:

Um, and it could essentially have two ways to go for what now I can, can

Speaker:

go to, who's calling the statement for, by execute and taking too long.

Speaker:

Then I can look up to the query, can look up to the execute

Speaker:

and goes through that all.

Speaker:

Or I can do something a little bit more smart because I, know this is just one,

Speaker:

one function that is, um, is so, um, hard.

Speaker:

So it should be able to sort by it, in theory at least.

Speaker:

Um,

Speaker:

but apparently I can't . Okay.

Speaker:

Um, then we need to go, uh, just click through.

Speaker:

So,

Speaker:

We just need to always look at where the largest results and that doesn't

Speaker:

take long and function execute again.

Speaker:

The parent function of that.

Speaker:

And now we see this query function is not called by much.

Speaker:

There's just as good default revision ID from content moderation.

Speaker:

Then they assess moderation state five item.

Speaker:

Then there's, get latest revision idea and get latest translation, get

Speaker:

latest translation affected revision in here, but it's kind of interesting,

Speaker:

but we see already our bugger is here, get later to revision ID.

Speaker:

So, um, and this is called by entity type is revisionable.

Speaker:

Um, and, um, this takes very long apparently.

Speaker:

Okay.

Speaker:

So, now we wanna want to look at, um, Oh, and this is it's even called by this

Speaker:

parent functions in certain contexts, but it can also be called itself.

Speaker:

So this would be the view from internal.

Speaker:

How do you get this nice graphical user interface, uh, on here?

Speaker:

Um, right now this is just a pull request, um, in the DDEV contrib,

Speaker:

uh, repository, it's a pull 128.

Speaker:

Um, and, this is essentially, it has a nice recipe of how you can set it up.

Speaker:

The first thing you need to do is in your DDEV config YAML file.

Speaker:

You need to add this web image, extra packages for each path, but

Speaker:

I've been told that in future DDEV versions, um, xhprof will be available

Speaker:

by default, similar to XD back.

Speaker:

So that's a great win ready for open source, because there's

Speaker:

something else, proprietary supported, out of the box, also.

Speaker:

Um, then we copied yourself a folders to our data folder.

Speaker:

Essentially you just follow this recipe and then for the application,

Speaker:

um, I've used the WordPress way because some of the, uh, Drupal way,

Speaker:

the Composer way didn't work for me.

Speaker:

So just download this PHP profile here and required both those files

Speaker:

and there I was up and running.

Speaker:

So let's take a look at how this looks.

Speaker:

Um, this is essentially just two lines of code edit here and that's it.

Speaker:

By the way, I did the same mistake as we originally did

Speaker:

with the old New Relic thing.

Speaker:

Um, I didn't had Redis enabled first.

Speaker:

Um, so I, uh, quickly mitigate that by adding Redis here as well.

Speaker:

Um, we have a standard Pantheon information, Uh, just because, uh, before

Speaker:

it was kind of screwing up my database statistics because was having way more

Speaker:

load the database than I expected.

Speaker:

So I'm not going to do that.

Speaker:

The other way, how we could've

Speaker:

... That was setting PHP, correct?

Speaker:

That you were just looking at,

Speaker:

uh, penalty questionnaire again?

Speaker:

That was settings.php that

Speaker:

you were looking at.

Speaker:

Yeah, this is settings PHP.

Speaker:

So we are looking at, um, site's default settings, PHP,

Speaker:

then settings page from Drupal.

Speaker:

And this is just where I am.

Speaker:

Um, just before DDEV includes its settings.

Speaker:

I put this auto load here.

Speaker:

I still hope, um, that this in DDEV itself we can do some little

Speaker:

tricks of either using pre-load.

Speaker:

Or adding it automatically to settings DDEV PHP, if you do, xhprofs on and off

Speaker:

like that, you can enable it by default.

Speaker:

And, um, then we can have some little, um, helper script.

Speaker:

Um, that is not a full-blown graphical user interface if all you

Speaker:

want out on traces, essentially.

Speaker:

Um, so, um, but this is a great solution that works very well with Goose.

Speaker:

Our Goose Attack is still running, um, as also printing

Speaker:

statistics from time to time.

Speaker:

Um, but not, we have not yet seen the, um, the maximum, so

Speaker:

I'm gonna stop this here now.

Speaker:

And, uh, once Goose has stopped, we can see clearly see here we have an

Speaker:

anon request and it took 23 seconds.

Speaker:

So despite in theory, um, this thing should only be affecting,

Speaker:

um, um, authenticated users because cricket is only there.

Speaker:

Apparently we have traces where, um, this is affecting anonymous users as well.

Speaker:

So that's pretty interesting and makesa critical bug, even more critical.

Speaker:

Um, this is something that Goose found this didn't show up in that

Speaker:

trace as clearly in, in New Relic.

Speaker:

Um, and, um, uh, when we are testing on Pantheon not on a local, we also have the

Speaker:

CDN, which is pushing a lot of load off.

Speaker:

So it wasn't as prevalent here, but Goose can clearly show us, um, that the

Speaker:

authenticated user here in this node called for posting the authentication,

Speaker:

getting the authenticate, um, edit form, uh, there's all performance problems.

Speaker:

Yeah.

Speaker:

So, um, this is, um, really, really nice of Goose of, of showing us all

Speaker:

this and, uh, We can see that even that hour average is still, uh, pretty okay.

Speaker:

Uh, for many things.

Speaker:

Um, but it's a maximum where we have the real problems and,

Speaker:

uh, Goose shows it directly.

Speaker:

So, um, if you, hadn't a nice tool like Goose, um, how would

Speaker:

you go ahead in doing this?

Speaker:

So you would go to quick edit, entity view alter and, uh, you would need to know.

Speaker:

and if I remember correctly, but even saw this, as this latest revision

Speaker:

in the New Relic, uh, screenshot.

Speaker:

So, um, okay.

Speaker:

I mean, now it's, it's this, this latest revision, what's

Speaker:

this latest revision even about?

Speaker:

there was something I asked and even questioned core about.

Speaker:

Core has the idea of forward revisions.

Speaker:

So you have a published revision and any revision created afterwards is a

Speaker:

so-called forward revision, which usually are drafts of the content, et cetera.

Speaker:

But if you are, in this mode, and apparently you want to quick edit.

Speaker:

you never want to edit, um, the published version, but only the

Speaker:

latest revision, essentially.

Speaker:

Even if what you are seeing right now is the published version because

Speaker:

it hasn't been published yet.

Speaker:

So, um, this is all a little bit complicated because Core, um, except

Speaker:

if you use workspaces spaces, which you really should, because then it's all

Speaker:

clear of what belongs to what, because then if you change the workspace,

Speaker:

It's clear you are editing in this workspace and even changed to publish.

Speaker:

It's clear you are editing published, but here it's what

Speaker:

would this edit essentially?

Speaker:

What is this edit the draft?

Speaker:

Clear to me?

Speaker:

What you just said was confusing.

Speaker:

Are you saying that we're editing the currently published or editing?

Speaker:

Not currently published

Speaker:

right

Speaker:

now?

Speaker:

The trick is whenever, um, we are looking at a live version, then this quick, quick

Speaker:

edit link does not want to show, because it would be confusing if you edit it.

Speaker:

And then it's a completely different title in the edit mode then in this.

Speaker:

so quick edit.

Speaker:

It's only going to show up if you're showing the, the last

Speaker:

even unpublished, um, revision?.

Speaker:

Is that what you're saying?

Speaker:

Yes.

Speaker:

If you looking at the latest revision ID, essentially.

Speaker:

Just then will the quick edit link show up.

Speaker:

But there could be editors that can't access drafts or a workflows, or like that

Speaker:

I'm only seeing like the live version and so for those this

Speaker:

link just should not show up.

Speaker:

So Core has , this idea of a latest revision ID is

Speaker:

essentially the latest draft.

Speaker:

And, um, the latest revision ID is essentially the maximum revision ID.

Speaker:

At least if you don't put translation into the picture, because then

Speaker:

it gets even more complicated.

Speaker:

It's a maximum revision ID, uh, in the note revision table, grouped by node ID.

Speaker:

So, uh, in theory its, as simple as, um, this.

Speaker:

Its not the count, but it's a maximum of vid.

Speaker:

So here we see, um, what the, um, maximum revision ID, uh, is, uh, based on the node

Speaker:

ID, maybe we should put the node ID in between, so, and because I've created a

Speaker:

lot of revisions once and deleted them.

Speaker:

Uh, we have pretty high.

Speaker:

node IDs by now.

Speaker:

Um, but yeah, this is essentially, uh, what Core expects to do.

Speaker:

So what we can also do is we can write a little test, um, now that, uh, Goose

Speaker:

is, is no longer here, um, we can write a little performance test of

Speaker:

essentially just running this part.

Speaker:

So, um, what I like to do is I called it SCR dot PHP.

Speaker:

And the reason is because I can just do DDEV SSH.

Speaker:

And I go to the directory where I have set, and then I do Drush

Speaker:

SCR, which is short for script.

Speaker:

And then I execute this little script and I can even put a time

Speaker:

before it, and after sometime, uh, we'll be having a result of what the

Speaker:

latest revision ID for node 16 is.

Speaker:

Uh, and it's correct.

Speaker:

That's certainly correct.

Speaker:

Um, we can see that here.

Speaker:

Um, but we can also see it took 12 seconds and this one nicely enough

Speaker:

is also showing up, uh, was the URL of slash um, this could be improved

Speaker:

a little bit of like, for example, showing the command line parameters.

Speaker:

If it's on the command line, we can see it took, um, 12 seconds here.

Speaker:

We can take a look at the one and, uh, nicely enough.

Speaker:

Of course, it's data may statement for back again.

Speaker:

Uh, what's making the problem and get latest revision ID as we already know.

Speaker:

So, yeah.

Speaker:

Um, so we found our troublemaker essentially.

Speaker:

Um, now we need to find it in the code.

Speaker:

Um, now that we've looked at it from all the stages.

Speaker:

so such a script, it's very, very useful for testing this because you don't want

Speaker:

to, especially with complex scenarios, you want to really just performance test,

Speaker:

this part that is so slow.

Speaker:

You don't want to performance test, everything essentially.

Speaker:

So, um, from Quick Edit, it, we would then, uh, searching for we

Speaker:

would to go to its latest revision.

Speaker:

Um, For the anti interface find that either via IDE or via grepping.

Speaker:

Again, this is calling get latest revision ID here.

Speaker:

And, um, and from there, there we go, go further to the front

Speaker:

NT storage space, whereas the get latest to revision ID is implemented.

Speaker:

Um, and this is essentially an entity query, which is having the

Speaker:

special, latest revision part.

Speaker:

And this is what introduced the problem essentially.

Speaker:

Um, and, uh, for the get latest translation effective to revision ID.

Speaker:

They have some other special key here, but they're essentially doing a range in

Speaker:

there doing a sort, uh, for the revision.

Speaker:

Um, and the reason for that is in using all of the revisions instead

Speaker:

of latest revision; in the original quick edit code, use this part as well.

Speaker:

They also use this kind of sort, et cetera, and it was fast and it was nice.

Speaker:

Just when Core introduced this generic interface, that is

Speaker:

when things got really slow.

Speaker:

And the reason for that we see in the last part where the actual

Speaker:

query is, and we see there's a left join, a self join on the table.

Speaker:

Where we look at all the revisions being smaller and then being as null.

Speaker:

And that prepared that for us here.

Speaker:

Um, and again, we can see this takes like a long time, uh, like five seconds here.

Speaker:

And the reason why this is so slow, um, this kind of query is, and, um, in

Speaker:

this case, and explain doesn't even, even give us this information because

Speaker:

it says, uh, using where, using index not exists even, um, is, um, because we

Speaker:

cannot really optimize based on the idea that we have the node ID already and,

Speaker:

uh, essentially its growth quadratic.

Speaker:

Based on the number of nodes and the number of revisions, because the self join

Speaker:

on a table and 10 is null, is performance wise, not a good idea, but in problem

Speaker:

is, uh, this max revision ID would also not work, uh, due to the fact, um, that

Speaker:

we really, really, really want to filter this whole query, uh, based on that.

Speaker:

So the best compromise we came up this essentially, instead of having all this,

Speaker:

um, this, um, left join is to using, uh, instead, um, uh, just, um, an inquiry.

Speaker:

And then when we select, um, again, the max vid from node revision.

Speaker:

Limit leads.

Speaker:

Okay.

Speaker:

So, and, um, I do the, the mistake.

Speaker:

Yeah.

Speaker:

Sorry.

Speaker:

oh,

Speaker:

sorry.

Speaker:

Yeah.

Speaker:

Uh, and this needs again to be grouped by node ID, and now we have a result

Speaker:

and I'm gonna explain, um, says that at least for, not too many node IDs.

Speaker:

Now we are based on the number of node IDs.

Speaker:

Um, this view can even be materialized and using an index for group by, and

Speaker:

that's already much, much better.

Speaker:

It's using a complexity of the number of nodes, but it's no

Speaker:

longer a quadratic complexity.

Speaker:

And that's really, really important.

Speaker:

Um, the problem is, um, this is still not making any use of the

Speaker:

fact that we know the node ID.

Speaker:

So essentially, um, if, uh, we give this information to the sub query, which we

Speaker:

cannot generically do, um, at, sorry, not base table, that's not now in here.

Speaker:

Um, then, uh, it can just use an index.

Speaker:

It doesn't even need to use group buy anymore because, well, there's only one,

Speaker:

one NID, so it can essentially change this and then select tables optimized the way

Speaker:

it's just, uh, materialized very directly.

Speaker:

So, um, and this is how essentially we, um, a solve this, um, for the core issue.

Speaker:

Um, it's um, let me quickly to change to the branch or let's take a look at the

Speaker:

dif.

Speaker:

Probably

Speaker:

easier.

Speaker:

So what we are doing for gate late to vision ideas, essentially, we put

Speaker:

it back to what it was originally.

Speaker:

We just get all the revisions.

Speaker:

We put a range of 01 on it and then we sort by the revision

Speaker:

ID in descending order.

Speaker:

And that's fast because that's what the database can very easily do.

Speaker:

It's as similar as just doing the max.

Speaker:

Um, the other part is instead of using this join and this wasn't my idea,

Speaker:

but someone else essentially had the idea that we access as a sub query.

Speaker:

So we essentially say, uh, the revision fields should be in this max group

Speaker:

by expression, which we just ran, uh, essentially this, uh, group by NID.

Speaker:

And, um, this isn't optimal.

Speaker:

It would be much better to have a, have a nice mapping table, which you could

Speaker:

join for where Core always tracking the latest revision ID of, of any revision.

Speaker:

Um, but this is at least much, much better, uh, than we had before.

Speaker:

Yeah.

Speaker:

And this is, not core patching in real time, but at least quite

Speaker:

explaining, explaining your corporate and how we can use Goose

Speaker:

to, uh, really help us streamline things, plan performance problems.

Speaker:

And, uh, yeah.

Speaker:

Even prove that this is not only affecting our same authenticated

Speaker:

users, but also anonymous users due to whatever, but this is Umami standard.

Speaker:

So, yeah.

Speaker:

Are you able to run the load test again with that same database, but with

Speaker:

the patch applied so we can see that

Speaker:

it doesn't have that state anymore.

Speaker:

That's a very good idea.

Speaker:

Let's do that.

Speaker:

Okay.

Speaker:

We have signature.

Speaker:

Um, I mean, it's really impressive to me and, you know, makes me understand

Speaker:

why we have such a great business.

Speaker:

You can talk about how you ran that load test and you saw that there was

Speaker:

a problem, but it took a tremendous amount of creative problem solving

Speaker:

to actually find the problem.

Speaker:

And then you need to know a ton about performance tuning

Speaker:

code to fix the problem.

Speaker:

Uh, so you know, it's a pretty multilayered, uh, challenge when

Speaker:

you're, when you load testing here.

Speaker:

Yes, it is.

Speaker:

Um, performance tuning is not for the weak.

Speaker:

It certainly is.

Speaker:

It's it's, it's not the problem is you never know.

Speaker:

Uh, if you find a problem.

Speaker:

Um, if you can fix it, if what kind of, um, um, structure you need and, um, with

Speaker:

many of those parts, um, yeah, performance problems can be really, really tricky to

Speaker:

find and exploit, but these are just tools that can help make it easier, et cetera.

Speaker:

And then even Core had the right idea.

Speaker:

Um, but, there was feedback that the group bike ferry wouldn't be better and

Speaker:

we need something much, much better.

Speaker:

And so a good solution was, um, uh, delayed due to a perfect

Speaker:

solution, not yet being there.

Speaker:

Um, and that's also the little bit, and here you need really need to understand,

Speaker:

um, if you're still studying, study your big 0s, they're so important

Speaker:

to know if something is quadratic complexity or linear complexity,

Speaker:

it makes a world of difference.

Speaker:

So, yeah, and hopefully this is fast now.

Speaker:

So let's see, um, what a longest wait time is.

Speaker:

Yep.

Speaker:

So longest wait time by now is 2.4 seconds.

Speaker:

And we've been running already for quite a while, so we can

Speaker:

assume, uh, this is fixed.

Speaker:

So then if you also canceled the Goose load test, um, you

Speaker:

shouldn't see any of those really

Speaker:

long.

Speaker:

Yep.

Speaker:

Probably.

Speaker:

You can also tell that into the controller and do metrics to not stop it if you want.

Speaker:

Yeah.

Speaker:

Sorry, exited already, but yeah, there's a

Speaker:

very cool,

Speaker:

normal max 23 seconds.

Speaker:

Is that pretty common, you know, that this sort of pattern, you know, the complexity

Speaker:

of, of finding, um, I'm sure it's hard to say, but is it like 50, 50, 80, 20?

Speaker:

Like what percentage of the time is a problem?

Speaker:

Like really obvious and easy to fix and, you know.

Speaker:

Most time you find a root cause at some point, um, the trickiest performance

Speaker:

problems is if nothing is slow, um, but just the whole thing is slow because it's

Speaker:

just for example, too many layers deep.

Speaker:

Um, then the best thing you can do is to add really nice caching, um,

Speaker:

because, but if there's something that is really slow, then it usually

Speaker:

can be fixed in some way or another.

Speaker:

Awesome.

Speaker:

Fabian, Jeremy.

Speaker:

That was really cool.

Speaker:

thank you for walking us through that.

Speaker:

for the folks listening, please be sure to check out our other

Speaker:

Goose talks at Tag1.com/Goose.

Speaker:

we have lots of blog posts on how to use the tool, how to find

Speaker:

performance problems and how to fix it.

Speaker:

Like today's talk, uh, if you have questions about Goose, please post

Speaker:

them to the GitHub issue queues.

Speaker:

If you'd like this talk, please remember to subscribe and share it out.

Speaker:

You can check out our other Tag1 talks at tag1.com/tagteamtalks.

Speaker:

And as always, we'd love your feedback on this episode, as well

Speaker:

as topic ideas for future episodes, you can write to us at ttt@tag1.com.

Speaker:

That's a Tag1 Team Talks at tag1.com.

Speaker:

And Fabian, thank you so much for joining us.

Speaker:

Uh, Thank you to Pantheon for providing hardware to test these Drupal instances

Speaker:

and thank you to everyone who tuned in really appreciate you joining us.

Links