Tag Archives: DBA

Quick and Dirty UPDATE STATS Job with T-SQL and PowerShell

When performance issues rear their pretty little heads I tend to look at the simplest things last. I am the person who is looking at the code first – because in many cases, if your code was better we wouldn’t be having this discussion. Whether it’s a series of nested views or one single user-defined function that doesn’t play well with the optimizer, there is usually something in the code itself that is contributing to the performance issue.

But what about the times when it isn’t the code or you can’t touch the code (I’m looking at you NHibernate…and all of your red-headed step-child offshoots and fellow ORMs). What about the cases when the poor performance situation only occurs periodically, or once a week on the same day, every week, and while you can see the slow-downs in your queries, and all of your waits and plans, the source of the problem isn’t any particular smoking gun – it’s lots of guns, firing off at random times, and the casualties are in your interfaces that depend on the data from the queries.

The one thing I don’t immediately go to is UPDATE STATS – and I should be looking at this. It’s something easy to run and if you still have issues, you have ruled it out.

Over estimation on rows = bad times for the optimizer.

In this case, I ran UPDATE STATS on one table in particular and immediately saw a marked improvement in my test query.

Ahhhh….much better.

This morning I ran the same query and things were back to where they were with performance. This was somewhat disheartening, but the troubleshooting must continue.

But what about auto-update stats? Is this on? Yes, but auto update statistics is only going to update based on a percentage sample of the records in the table. If your table contains millions of records, you may not be updating your statistics sufficiently to see an improvement. I am updating statistics with a FULL SCAN of the table. For larger tables, this may take some time, but you may also see a difference in your execution plan versus only updating statistics based on a sample.

In an effort to rule out whether or not statistics are definitely a factor, I want to UPDATE STATS on all the tables in my query, and at a specific time – sometime the day before we expect our slowdown to occur. I also want to be able to easily see how long the process ran and the duration of the update for each table. I could write all this to a table, and maybe I will do this later, but viewing this from job history is what I want right now – it is easy to look at and easy for everyone else to find and see as well.

Creating a job with multiple steps can sometimes be a bit painful and tedious, if the job is going to have A LOT of steps. With PowerShell, a server name and a database, I can dynamically create this job.

The result from this is a job that will update statistics on your list of tables, one at a time. All you have to do from there is give it a schedule.

All those steps…courtesy of the magic that is POWERSHELL!

For this script, I was getting my list of tables based on a specific query that was problematic. If you are doing this for a stored procedure, you can query for the list of dependent tables instead, and instead of hard coding the tables in this code, pass in the name of your stored procedure and get the list that way.

I hope if you have a situation where some one-off stats updates are needed you find this script to be useful. As always, comments and feedback are appreciated. πŸ™‚

 

Blocking – I Saw What YOU Did and I Know Who YOU Are!

When I was growing up I remember my mom talking about an old, scary, movie that she saw when she was young. In the movie some teenagers were making prank phone calls saying “I saw what you did and I know who you are.” One of the calls happened to be made to a guy that just killed his wife. Joan Crawford played a woman that was romantically inclined to said murderer. She eventually meets her demise when he stabs her because she knew about the first murder.

How every DBA feels when there is blocking

While blocking in SQL server might not be a felony offense (it isn’t…but it should be – WHO’S WITH ME?) as the DBA you not only want to know what is being blocked, but also who is doing the blocking and what in the H-E-Double-Hockey-Sticks they are doing.

At SQL Saturday Orlando I talked about this very thing and the query I defer to for the information.

IMA GONNA HUNT U DOWN!

It might look complicated but it is actually very simple – query sys.sysprocesses with a cross apply using the sql_handle to get the text of the query, and then an outer apply with the same query again but you are joining to the blocking spid so you can get the text for the query that is doing the blocking. Beyond that, you can filter on various columns and refine your output

WHY ARE YOU BLOCKING YOURSELF? WHY ARE YOU BLOCKING YOURSELF? WHY ARE YOU BLOCKING YOURSELF?

Of course, I know you can’t run this without the code (and I know that’s why you’re here…because I SAW WHAT YOU DID!)

If you are looking for a good way to troubleshoot blocking I hope this helps. If you have some folks running queries that are making you stabby, run this, find out what is going on, and then remove their access try to help them so they aren’t making you stabby any more. Then tell them they need to buy you a beverage because they are still alive.

UPDATED: PowerShell – WE HAZ YUR ACTIVE DIRECTORY RIGHT HERE!

I discovered something last week – I had not blogged about little things that I thought I had blogged about. What the heck does that mean? It means that I tried to reference my blog for something because I thought “I totally blogged about that”…and found out that was not the case.

Starting now, I am fixing this situation. There was something that popped up today that called for a PowerShell script and the Get-ADGroupMember cmdlet – get a list of users from a list of groups. Some users are in there more than once so this needs to be a distinct list, unless you are into manually cleaning up things like this, and then I will be sad for you. Because that is kinda sad.

I originally wrote a script with two arrays (one for the initial list and one for the de-duped list of users), but even though this is quick and dirty, that was a little too dirty. Enter the Group-Object cmdlet – it takes this list of names and groups them. No black magic this time. Just a cmdlet, that comes baked into PowerShell giving me what I need.

What? You wanted the code too? Oh, OK.

There you have it – quick, dirty and to the point. Enjoy. πŸ™‚

UPDATE:Β Mathias Jessen tweeted a one liner for this….so no need for the one array! Woohoo!

I was trying to do this but was also just trying to get it done, and if in doubt, I slap things in arrays. Thanks Mathias!

 

Speaking – SQL Saturday Baton Rouge!

This Saturday will mark the fourth year that I will be speaking at SQL Saturday Baton Rouge at LSU.Β Out of all the SQL Saturday events I have participated in since 2014, Baton Rouge is one of the few that I have been to every year since. Houston is another one…and actually, these may be the only ones only because they have had an event every year.

Not only am I speaking on Automation with PowerShell and Deadlocks and Blocking, but I am also participating in a panel discussion on Careers in IT. I’m excited to be invited along with some of the other speakers to be a part of this. Looking back on my own school days, I knew I would have a career in IT, but little did I know I would detour from a path in development to the world of SQL Server, and becoming a DBA.

How do they make this happen? Work. Lots of hard work. After helping with SQL Saturday Dallas 2015, joining the NTSSUG board in 2016, and then having an organizing role in 2016, I found out how much goes into the planning of these events. If you have attended a SQL Saturday or you are going to in the future, be sure to say “THANK YOU!” to all the organizers and sponsors. If you want to get more involved in the SQL Server community, SQL Saturday is a great way to do that – just show up at the event, find an organizer and tell them that the SQL Kitten sent you to be their humble servant volunteer for the day…or you could just say you want to volunteer and leave out the other stuff because it might make it weird.

SQL Saturday NEW YORK CITY? Get a rope!

This weekend I will be speaking at SQL Saturday New York City. My last time here was two years ago for the last SQL Saturday here and I had a great time. If you are in the area come on out for some awesome speakers and a great day of learning!

Anyone remember this? No? Just me, huh? Ok.

Blogging reboot and PowerShell – Get System Information

After whining about not blogging enough, I am going to do something about this. Whether it is PowerShell or SQL, simple or complex, I know that others can benefit from my knowledge and expand their skill sets.

And you are like “That’s great Amy…where’s the PowerShell we are here for?” Ok, Ok….keep your drawers on! πŸ˜‰

Recently I was tasked with gathering the system information from all of the servers at a client. Another opportunity for some PowerShell dominance.

With this script you can generate system information files and save them to a specified location. It makes sure a connection can be made to the server first, and then outputs the file. The files are created one at a time, so if you pass in a longer list of servers, you shouldn’t crash your machine. From my testing, this will take some time to run as these files don’t output quickly. Despite that, the output is worth it. This can be modified to pull your list of servers from a file or from a Central Management Server (CMS) instance.

Hope you find this useful. Happy PowerShellin’ πŸ™‚

Podcast with SQL Data Partners

Recently I had the pleasure of meeting Carlos Chacon with SQL Data Partners and found out he had a podcast (I know…I live under a rock…with cats). He asked if I was interested in being on said podcast – ummmm…yes, please! Click here (Episode 33) if you wanna go listen to the podcast or here if they get all up in your grill about streaming stuff at the office and you wanna just read the transcript. Thanks to Carlos for the opportunity! Enjoy! πŸ™‚

T-SQL Tuesday #73 – Those Darn Elves

t-sqltuesdaySince I have not blogged in a while, and I saw it was T-SQL Tuesday, I thought I would participate in this FOR THE FIRST TIME EVER! This month SQLBalls asks us if our SQL Servers are on the naughty or nice list. Since I have recently transitioned to a new role, I am still getting familiar with the servers and the environments I am working with. The servers of my past are but a memory, but a fresh one. I cannot even begin to think of all the different “naughty” things that were done on those servers, and how some days it felt like a losing battle. How can you fight a security or code change when you have been over ridden by management, only to have that change come back and bite you weeks or months later? The “I told you so” you might feel like popping off will fall on deaf ears and you will just be stuck fixing the problem…until it gets to be too much and you decide it is time for action.

I have done this before. The day had been a long one and I thought I was finally going home when I got drug into a call on a data issue – something was changed that shouldn’t have been. When and by whom? I didn’t know or have any way of finding this information (days later when I tried to get a backup that was several months old to validate this data from when it was deployed; there was no backup – all all – but that is a completely different issue for another time). I had my suspicions but no proof. Could have been a naughty developer elf logging in with an elevated SQL account they knew the password for. Changing the password? “Out of the question” they say. It is everywhere.

If this were the only event that had happened THAT DAY. It wasn’t. This one was production (hence the conference call). The others (yes, more than one) were pre-production. Messes that had to be cleaned up because the developer elves thought they knew better than the DBA and that they could do it on their own. This production issue was the last straw. Elves were running a muck and had to be reigned in, and they weren’t going to like it.

All the elevated permissions in the pre-production servers – gone. I didn’t care if it was a DEV server. Am I the meanest DBA in the world? So says some. Scrooge? Well, if you are into name calling and want to go there, then ok, but I get to call you names too. In this case I did not care – I was fixing things that were only broken because someone abused a privilege. It should also be said that there was some relation in the names off all the tables involved with all the issues that occurred on this day.

If I had to wag my finger and any naughty part of the SQL Server instances it would have to be at security…and I am partially to blame. It can be difficult to keep up with all the changes that happen across a large environment when it comes to assigning permissions, and if you have more than one DBA, the situation is compounded by the fact that you might not always know what the other is doing and vice versa. They might grant something that you would otherwise veto for cause. You might take care of a permissions issue one way when they would handle it differently.

While I worked on some Powershell code to pull back users from specific AD groups and incorporate alerts for some of those groups, sadly the bandwidth was not there to fully roll this out. I did however create some triggers that would send email alerts when a change was made at the server and database levels, and I made them nameless and encrypted.

 

 

The lack of a name for each of these is intentional, as is the encryption. The last thing I wanted was someone seeing these ans what they were doing, and if they had permissions to do so, disabling or dropping them to avoid having their nefarious behavior tracked. Even better would have been to put additional triggers in place to prevent the dropping of these no matter what, but I decided not to go there.

Note there is nothing there for the name for each of these – this is courtesy of the devious mind of Rob Volk. He might have too much time on his hands but this is pretty darn crafty. What you name these triggers is up to you but you have to MAKE NOTE OF WHAT THE NAME ARE!!! When I did these they were a combination of a few tabs and spaces – like “space space space tab tab” but with those actual characters. The result looks like this:

 

triggers_noname

It should go without saying use this code at your own risk and always thoroughly vet and test anything before applying it to a production environment.

If this helps further cement my meanest DBA creds then I guess I am doing it right. Sometimes the elves developers can get out of hand and it is up to Santa the DBA to make sure they know they are being watched.

 

3/12 – Webinar : Beginning Automation with Powershell

Anything I can do to reduce the time I have to spend dealing with otherwise time-sucking tasks gives me a happy. Currently, where I am, I have reduced our daily deployment process to a single Powershell cmdlet with a log file review. All the steps that normally went into each deployment (applying SQL scripts, data updates, updating the TFS task, sending an email that the deployment is done) are handled by Powershell, and the process accepts multiple deployments at the same time. As long as there are no errors in the log file for a deployment, there is nothing more to do. There’s an error? Run the same cmdlet with the rollback option.

I am still working on improving this process, from trying to get the error handling to work the way I want it to, to incorporating the automation of check-in to source control of the changes that were just applied. It is a work-in-progress that will continue to reduce the amount of time spent on what would otherwise be manual tasks.

This work led me to the idea for a session on Powershell and automation of processes. Join me and Pragmatic Works on Thursday, March 12th from the comfort of your own desk at 11am EST/10am CST for Beginning Automation with Powershell. This demo-heavy session will go over some different Powershell cmdlets and automation concepts that you can take and build upon in your own environment. Powershell is a tool that no DBA should work without. A development background is helpful but not necessary – what DBA would not want to spend a few hours writing code for a process that will save them from having to do a repetitive manual task over and over again? Automation frees up time for us to do other projects that we want to spend time on and grow our skill sets.

Have questions about this? Leave a comment below. See y’all there. πŸ™‚

#SQLSatPhoenix This Weekend!

Both the SQLSat Austin and Albuquerque events were great – now on to Phoenix. I will be giving two sessions there – Deadlock, Block & Two Smoking Barrels: Breaking Down Blocking and Deadlocks and Making the Leap from Developer to DBA.

The first of the two is a new session for me. When it comes to blocking and deadlocks, I like to be able to identify the cause (and sometimes the culprit) and have the information needed to resolve the issue. Some methods are easier, while with others, you also have to think about the footprint of the troubleshooting itself.

With Making the Leap from Developer to DBA, this session has evolved with time and my own experiences. Anyone who is looking to transition to being a DBA full time should attend to get a real-life perspective on migrating into this role and some of the things you can look forward to.

On top of all this, I have also been invited to be part of the WIT Panel. Come join my self and some other amazing women during lunch as we share and discuss confidence and our experiences in IT.

I’m looking forward to another great SQL Saturday as well as meeting everyone in Phoenix. See y’all there! πŸ™‚