Tag Archives: sqlpass

(UPDATED) PASS Summit 2017 Session Data (You know you want it…so go get it!)

UPDATE #1 –
The following changes were made to the PowerShell script for the PASS Summit 2017 session scrape:

  • Abstract field is now varchar(max). ‘Cause precons get all the chars.
  • Added SubmittedSession column.
  • Upper bound increased to include precons.

UPDATE #2 –
Before I could get update #1 posted, it appeared all the general sessions and lightning talks were removed from the submissions page and replaced with submitted precons. Don’t fret – the PowerShell script will still grab all the submitted sessions (for now). There were also some more precons added so the upper bound has been changed yet again.

 

The deadline for PASS Summit submissions came and went. Either you got yours in or you didn’t. What does this mean? Time to gather the data.

Go get it! You know you want to!

Never having done a web scrape before, this was the perfect subject for my first time – grabbing all the sessions submitted to PASS Summit 2017…and doing it with PowerShell! Here is the script I used for this. I have accounted for the following:

  • Apostrophes (aka single quote). They will break your insert unless you have two of them, and for some reason, people seem to use them all over the place.
  • Formatting the string data for insert. No, your data will not magically come out right in your insert with single quotes so you need to add them.
  • Additional ID and deleted fields.
  • Speaker URL and ID. Will be using this to scrape speaker details later.
  • Accurate lower and upper bounds. These were arrived at by trial and error (you’re welcome), as well as the clean up of the data I scraped. More on this later.

Now for the code –

 

There will also be some webids that error out – this means the session doesn’t exist for that specified webid. You will see an error in the output to the PowerShell window, but the script will keep running.

Once you run the script and it completes, you will have your table of session data. You will notice that there are more sessions there than have been submitted, according to the website. If you do a count of the sessions by speaker you will also find that there are more than three sessions (the maximum allowed) for quite a few speakers. Why is this? You are pulling from a data source that contains sessions other than the ones that were submitted to Summit (apparently). This means you will have to clean up the data – I included the deleted field for this purpose. Look at the duration and filter for the 75 minute and 10 minute sessions and update your deleted field accordingly.

After this, you will have sessions that meet the duration, but you will find the speaker is not listed on the submission site when doing an advanced search. Clean the data some more.

Are you done cleansing your data yet? You might think so, but you aren’t. You see sessions that look like they were submitted for Summit but are not on the site. These appear to be sessions that were created and saved, but not officially submitted. You can rule out quite a few of these if you look for speakers that have more than three sessions. Get those counts, check the website for those speakers, and clean your data.

“Good lord, am I done yet?”

No. No you’re not.

There will be speakers that have three sessions in your table but only submitted one or two. This sounds worse than it is, but it is a little painful. How do you do this? You’re going to search the speakers with the advanced search on the website. Do these in groups based on the number of sessions a speaker has, ordering by speaker name. You can exclude the the speakers you have already reviewed in the previous clean-up steps, deleting their names from your list as you go. When you find a speaker with three sessions in your list and fewer sessions submitted, mark the session that was not submitted as deleted. Next, do the speakers with two sessions as they may have only submitted one. Finally, look at the speakers with one session as there may be ones that didn’t officially submit anything.

“But what about the code for the clean-up?”

No. You know how to do this, and may write it differently (and better) than me. My code was ugly and I don’t feel like cleaning it up and posting it here. I gave you the script to grab it all. Stop complaining or I will send you to time-out.

The speaker info grab will be a separate blog post. With the script in this post you might be able to use the syntax and craft your own PowerShell script for this. Or you could wait for my blog on it.

Yeah…I haz the datas!

T-SQL Tuesday #73 – Those Darn Elves

t-sqltuesdaySince I have not blogged in a while, and I saw it was T-SQL Tuesday, I thought I would participate in this FOR THE FIRST TIME EVER! This month SQLBalls asks us if our SQL Servers are on the naughty or nice list. Since I have recently transitioned to a new role, I am still getting familiar with the servers and the environments I am working with. The servers of my past are but a memory, but a fresh one. I cannot even begin to think of all the different “naughty” things that were done on those servers, and how some days it felt like a losing battle. How can you fight a security or code change when you have been over ridden by management, only to have that change come back and bite you weeks or months later? The “I told you so” you might feel like popping off will fall on deaf ears and you will just be stuck fixing the problem…until it gets to be too much and you decide it is time for action.

I have done this before. The day had been a long one and I thought I was finally going home when I got drug into a call on a data issue – something was changed that shouldn’t have been. When and by whom? I didn’t know or have any way of finding this information (days later when I tried to get a backup that was several months old to validate this data from when it was deployed; there was no backup – all all – but that is a completely different issue for another time). I had my suspicions but no proof. Could have been a naughty developer elf logging in with an elevated SQL account they knew the password for. Changing the password? “Out of the question” they say. It is everywhere.

If this were the only event that had happened THAT DAY. It wasn’t. This one was production (hence the conference call). The others (yes, more than one) were pre-production. Messes that had to be cleaned up because the developer elves thought they knew better than the DBA and that they could do it on their own. This production issue was the last straw. Elves were running a muck and had to be reigned in, and they weren’t going to like it.

All the elevated permissions in the pre-production servers – gone. I didn’t care if it was a DEV server. Am I the meanest DBA in the world? So says some. Scrooge? Well, if you are into name calling and want to go there, then ok, but I get to call you names too. In this case I did not care – I was fixing things that were only broken because someone abused a privilege. It should also be said that there was some relation in the names off all the tables involved with all the issues that occurred on this day.

If I had to wag my finger and any naughty part of the SQL Server instances it would have to be at security…and I am partially to blame. It can be difficult to keep up with all the changes that happen across a large environment when it comes to assigning permissions, and if you have more than one DBA, the situation is compounded by the fact that you might not always know what the other is doing and vice versa. They might grant something that you would otherwise veto for cause. You might take care of a permissions issue one way when they would handle it differently.

While I worked on some Powershell code to pull back users from specific AD groups and incorporate alerts for some of those groups, sadly the bandwidth was not there to fully roll this out. I did however create some triggers that would send email alerts when a change was made at the server and database levels, and I made them nameless and encrypted.

 

 

The lack of a name for each of these is intentional, as is the encryption. The last thing I wanted was someone seeing these ans what they were doing, and if they had permissions to do so, disabling or dropping them to avoid having their nefarious behavior tracked. Even better would have been to put additional triggers in place to prevent the dropping of these no matter what, but I decided not to go there.

Note there is nothing there for the name for each of these – this is courtesy of the devious mind of Rob Volk. He might have too much time on his hands but this is pretty darn crafty. What you name these triggers is up to you but you have to MAKE NOTE OF WHAT THE NAME ARE!!! When I did these they were a combination of a few tabs and spaces – like “space space space tab tab” but with those actual characters. The result looks like this:

 

triggers_noname

It should go without saying use this code at your own risk and always thoroughly vet and test anything before applying it to a production environment.

If this helps further cement my meanest DBA creds then I guess I am doing it right. Sometimes the elves developers can get out of hand and it is up to Santa the DBA to make sure they know they are being watched.

 

My First Time Submitting to SQL PASS Summit – Part 1

I have now officially submitted to SQL PASS Summit 2015. Five sessions. All me (well, and a few others on the panel I submitted). The hard part is over…but how did this all happen?

My story starts back in the year 2013. My first time to attend a SQL PASS Summit. It was in Charlotte, NC. I didn’t know these were normally in Seattle. It didn’t really seem to matter too much either. I was a first timer. And they labeled me as such.

first_timer_2013

I didn’t mind. I knew I was a newb. I embraced it. I signed up for and watched the webinar for first timers put on by Denny Cherry. I also got a first timer buddy (aka Someone I can follow around like a puppy until I am comfortable enough and have found other suitable, like minded, people that will welcome me and allow me to accompany them to places that sell beer).

I met so many new people while at Summit that year and out of all the conferences I had been to in the past, this was by far the best experience I had ever had. One this that happened numerous times when talking to people was hearing the words “Have you thought about presenting?” I’m all “Wow…these people are nice…and encouraging…wait…presenting? Say what?”

Did they need fresh meat? A new crop of willing yet unsuspecting folks to throw their hat in the ring, to hopefully be selected, and dive head first off that cliff into something that might become an obsession that would challenge them in ways they had never thought of before? Maybe. Maybe they did. Maybe it was a little cultish. Well, pass the kool-aid.

I came home and gave it some thought. I really wanted to do this but did not know where to start. I felt like I had stuff to say. Stories to tell. Experiences to share. But where to start? I decided I would do a panel. I rounded up some experienced speakers for a particular SQL Saturday, created my session and submitted it. That was the easy part. I then decided we would have weekly internet video chats so that everyone could get acquainted, and I could get their take on how they saw the panel going. They were also able to provide me much needed guidance with my slide deck (something else I had never done before) and how the session needed structure. I took all this in and did my slides accordingly.

The time finally came for the SQL Saturday event and my panel session. I made the trip out there and met up with everyone. I attended my very first speaker dinner. I could not believe I was there and in the same room with some of the brightest and well known minds in the SQL community. This was it – my first taste of what would consume that entire year.

The panel went well. My fellow panelists and other patted me on the back for a job well done. Feedback was good. I was elated. I was hooked. I thought I was ready to handle what was up next – my first solo session. Just me and my slides. Another SQL Saturday. Another city.

To be continued…

#Summit14 Recap…and other things

I just got back from another amazing SQL PASS Summit – this time in Seattle. This year was different in that I knew people going into Summit. Last year I only knew some folks from Dallas and the alumni I was paired up with, Bill Fellows. I met a ton of new people last year and got to reconnect with them this year, and make new friends as well.

This past year also gave me the opportunity to make new friends from all the SQL Saturday events I attended and had the privilege to speak at – Las Vegas, Houston, Oklahoma City, Baton Rouge, Kansas City, Denver, and next month, Washington DC. Never have I traveled so much before but it has been worth it. I have learned so much – not just about SQL Server, but also about speaking and sharing knowledge and experience with others.

One thing that I am still in awe of is the amount of “nerd celebrity” you will meet at SQL PASS Summit and the SQL Saturday events. I am one of those people who could care less about the idea of meeting the typical Hollywood celebs, but put me in front some of the people at PASS Summit and I am starstruck. All that brain power in one location is something awesome to behold. And the sessions aren’t bad either.

A few things to note…

Thanks to Matthew Brimer for the opportunity to attend the Brent Ozar pre-con with him. It was awesome!

Thanks to the Midnight DBA’s. They never fail to impress.

Thanks to all the sponsors that threw all the amazing parties. My liver also thanks you.

To Denny Cherry and the invention of Speaker Idol – what an opportunity for those presenting as well as those of us who were just there to watch. Great idea! Thanks for making this happen.

To all those that belted some tunes at the various #sqlkaraoke events – you were awesome but it is good you have your SQL Server careers to fall back on. 🙂

Finally, to my #sqlbitches – you know who you are. Keep it real, yo <insert secret hand gesture here>.

See y’all next year…if not before.

SQL Saturday #309 – Oklahoma City

It is my second time at the SQL Saturday in OKC and my first time speaking at it. For the people who attended my session on Beginning Automation with Powershell, thanks again for coming. I hope everyone enjoyed the session as much as I enjoyed giving it.

As promised, the code is available for download below. If you have any questions, leave a comment here or reach out to me on twitter.

 

SQLSat#324 – Session Noms

Thanks to everyone who attended my sessions at SQL Satruday #324 in Baton Rouge. This was really a awesome event – thanks also to all the people that helped put it on and all the sponsors who helped make it happen.

I had a great time presenting and I think my demos went ok (for the first time ever doing demos). I also really appreciate the feedback since I want to keep on presenting. If you were in one of the sessions and have any further comments or questions feel free to leave a comment here and I will get back to you.

Also, I have gotten the scripts posted here and the download appears to be working now. Let me know if you have any questions about those. Enter the password below to access and download the scripts from Beginning Automation with Powershell.

Thanks again and see everyone next year!

SQL Saturday #324 – I’m headed South!

It is just a few more days until SQL Saturday #324 – Baton Rouge. I will be giving two sessions there – Making the Leap from Developer to DBA and Beginning Automation with Powershell. So looking forward to this event!

The last time I was in southern Louisiana I was barely in my 20’s and on a family vacation. I found that my normal routine for straightening my natural curls was no match for the level of humidity in New Orleans. This memory prompted me to check the weather for this weekend – yeah…this could get interesting.

See everyone there! 🙂

I’m Speaking – SQL Saturday #308

SQLSAT308_SPEAKING

This weekend will be my second time to speak at a SQL Saturday event, and the first time I will be doing it solo. Being a native Texan, I have to admit it is a little strange that I have never been to Houston before, but I cannot think of a more fitting reason to make the drive down there. There is also that whole Dallas vs. Houston thing….but that is an unrelated post that I think other sites have already covered ad nauseam.

My session this weekend will be on Making the Leap from Developer to DBA – a topic I am all too familiar with since I have lived it. Fulfillment of that desire to innovate and create things can be accomplished as a DBA, but that is only a small part of the role. The business depends on you to make sure data is accurate and available in a timely fashion. Developers depend on you for the same thing but also for things like deploying their code…and trying to prevent them from hanging themselves and you when you do it.

It is still not too late to sign up for the event. If you are in Texas or anywhere near Houston you should go…and come to my session. Hope to see you there. 🙂

Back from SQL Summit

Quick post – I am back at work this week from the SQL PASS Summit and I have to say I think it was the best conference I have ever been to. From the sessions to the people and the events, everything was crazy fun. I now see what people mean by #sqlfamily. Even though it was my first time, everyone was so welcoming that it didn’t feel that way. I am looking forward to more of the same next year in Seattle. Coming up soon is SQL Saturday #255 – Dallas (well, actually, it is in Arlington…hmmm) and then SQL Saturday #233 – Washington DC. Heck, I might even make it to the after events this time. If you want to connect with me at either of these events hit me up on twitter – @texasamy.

Optimizing SQL Server Performance in a Virtual Environment with Denny Cherry

T-SQL performance tuning – no differences between virtual and physical

Check host and GPU numbers – look outside of the VM at the host.

Remote Perfmon users – group you can be added to for viewing performance monitor information for the server; you can be added to this group on the server where your VM is hosted.

ESX Top – task manager for UNIX; command line utility. Check for CPU thrashing – looking for percent used.

Check host and guest for disk IO latency.

Balloon memory driver – should be enabled; used by host os to request memory back from the guest os; if this is turned off – prevents host os from paging physical memory to the hosts swap file; paging will occur and you will have random slowness that will be difficult to diagnose.

Reserved memory setting – How much memory you want and how much memory you really need to have (reserved).

Memory deduplication – looking for and stripping out duplicate instances of processes in memory; great for OS memory. Doesn’t work for all SQL Server – unless multiple SQL Servers have the same pages in cache.

Storage configuration options – IO is the same if the disks are physical or virtual; automatic tier adjusting technology if possible. Storage slow? Get faster storage ($$$). Keep OS, data, logs tempdb on separate disks if possible.

Storage deduplication – can greatly improve overall performance; deduplicating the OS virtual disks = save much less data to the array.

VMWare Paravirtualization Driver – optional driver for vSphere Virtual Machines; recommended for high IO workloads; config via VM Properties – select SCSI controller, change type.

Monitoring – look at more levels of the environment; SQL Server, guest OS, hypervisor, etc.

Counters –
reads/writes per sec – correlated counter at the host level
seconds / reads and writes
disk queue
page life expectancy
system processor queue
VM Disk
VM Memory

Timer inside the VM lies – ugly rumor and nothing more.