Category Archives: SQL Server

Back from SQL Summit

Quick post – I am back at work this week from the SQL PASS Summit and I have to say I think it was the best conference I have ever been to. From the sessions to the people and the events, everything was crazy fun. I now see what people mean by #sqlfamily. Even though it was my first time, everyone was so welcoming that it didn’t feel that way. I am looking forward to more of the same next year in Seattle. Coming up soon is SQL Saturday #255 – Dallas (well, actually, it is in Arlington…hmmm) and then SQL Saturday #233 – Washington DC. Heck, I might even make it to the after events this time. If you want to connect with me at either of these events hit me up on twitter – @texasamy.

Optimizing SQL Server Performance in a Virtual Environment with Denny Cherry

T-SQL performance tuning – no differences between virtual and physical

Check host and GPU numbers – look outside of the VM at the host.

Remote Perfmon users – group you can be added to for viewing performance monitor information for the server; you can be added to this group on the server where your VM is hosted.

ESX Top – task manager for UNIX; command line utility. Check for CPU thrashing – looking for percent used.

Check host and guest for disk IO latency.

Balloon memory driver – should be enabled; used by host os to request memory back from the guest os; if this is turned off – prevents host os from paging physical memory to the hosts swap file; paging will occur and you will have random slowness that will be difficult to diagnose.

Reserved memory setting – How much memory you want and how much memory you really need to have (reserved).

Memory deduplication – looking for and stripping out duplicate instances of processes in memory; great for OS memory. Doesn’t work for all SQL Server – unless multiple SQL Servers have the same pages in cache.

Storage configuration options – IO is the same if the disks are physical or virtual; automatic tier adjusting technology if possible. Storage slow? Get faster storage ($$$). Keep OS, data, logs tempdb on separate disks if possible.

Storage deduplication – can greatly improve overall performance; deduplicating the OS virtual disks = save much less data to the array.

VMWare Paravirtualization Driver – optional driver for vSphere Virtual Machines; recommended for high IO workloads; config via VM Properties – select SCSI controller, change type.

Monitoring – look at more levels of the environment; SQL Server, guest OS, hypervisor, etc.

Counters –
reads/writes per sec – correlated counter at the host level
seconds / reads and writes
disk queue
page life expectancy
system processor queue
VM Disk
VM Memory

Timer inside the VM lies – ugly rumor and nothing more.

Multidimensional Array from Array Data in Parameter

While I am not new to the world of coding (and not just T-SQL) I am somewhat new to Powershell. I cobbled together a monitor process that works with GoldenGate a while back and just recently made it less ugly and improved its performance. Beyond that, I am still on the green side.

Enter my current project – automating the SQL portion of our deployments using Powershell. Before this I was taking the list of items to be deployed and generating my own scripts to run using the Red Gate SQL Compare tool. Then came the day when I called one of the scripts from a developer in Powershell. There was no going back. I was hooked. I immediately saw how amazing this could be. And also how my work load would drop.

While I am doing my part in Powershell, there is also some other development going on with the end the developers use – in other words, things are in flux. I have code for the old way they do things and the new way. The new is better in that some of the things I need are built-in to certain aspects so I am able to grab pieces of information and get the things from that I need. With the old, this is not the case. Instead I am passing that information in through parameters.

I had the idea of passing in strings that would then be broken up and inserted into a multidimensional array. With that, I could loop through the array one element at a time, have all the things I needed for each deployment and just sit back and watch the magic happen.

Uhhhhh…did I mention I was green in Powershell?

I had another array already going so I thought I had this. I don’t know if it was because it was late in the day, or if I was a little too confident in my array abilities, but the darn thing was not happening the way I had envisioned. I set the code aside to revisit later with a fresh pair of eyes and a brain that wasn’t so full.

Fast forward to the following week – went back to the same code and FIGURED OUT HOW TO DO IT IN 5 MINUTES. Not only that, but it was far less code than I had previously attempted before.

Here is how I have handled this. Obviously, I would be replacing the text in the strings with information that is relevant to my deployments but the concept is the same. Once I have the array created, I have all the elements I need and it is just a matter of knowing where they are in the array and accessing them.

$strings = @("The cat in the"
,"hat ate a rat"
,"and then sat and"
,"got fat from the rat")

$array = New-Object System.Collections.ArrayList
$array.Clear();

foreach($item in $strings)
{
$array.Add(@($item.Split(" "))) | out-null;
}

for ($i = 0;$i -lt ($array | Measure-Object).count;$i++)
{
Write-Host $array[$i][0]
}

For the sample I am writing the first element. For my project the first element might be the server name for the deployment or some other relevant piece of information and it would be in the same place throughout the array. Since variables and their contents persist in Powershell I added the statement to clear the array before it is loaded. I am also wrapping the code up in a function to keep things tidy.

If you find this useful let me know. Enjoy. 🙂

SQL Saturday #223 – Mouth Wide Shut: Coherent Interviewing Part 2 with Sean and Jen McCown

How to find a good DBA

You are stupid until proven otherwise

Creating Job Wrecks (Requirements) – Be reasonable, honest, concise specific

Reading resumes – look for job hoppers; red flags things that are impossible; look for language that are trying too hard; Beware “experts”

After 10+ years – if they are listing things like index tables and other basic things, they are trying too hard – they should be doing a lot more than this.

Expert, Proficient, Master – key words to explore what they claim to be an expert in

Question – How are you continuing your education?

Phone screens – initial litmus; measure communication skills and basic knowledge; decide whether or not to bring them in.

Passive interviewing – less you talking, letting them think they should keep talking; when you are being interviewed you can turn this around.

Interview pre-questions – first impressions; small talk before the actual interview; can be used to trip them up later

Questions –

  • So what SQL sites you go to?
  • If you were going to consider yourself an expert in something what would it be?
  • What would you be if you were not a DBA?
  • What is it you like about databases?
  • Are you self-starter?
  • Do you follow any people on twitter?

You can get on a call with 70 users and say anything, but you’d better be right.

Before everything else – explain the format

  • What you’ll do
  • What you expect
  • If there will be labs
  • Helpful hints

Say the answer and shut up; If you don’t know say you don’t know.

Question types

  • Experience – How long have you been in databases? Are you an expert in anything? Have you put together an HA scenario?
  • Knowledge – what is the diffence between delete and truncate? What is a cte?
  • Probing – Tell me exactlly how that works

Labs

  • Short, discrete pieces of work (1-2 min)
  • Nothing that requires specific in-house knowledge
  • Something that truly common knowledge

Good: Labs incorportated into the interview

Bad: they have to sit in a room and do stuff

Contract-to-hire is your friend!!!

SQL Saturday #223 – SMO Internals for High Performance Powershell with Ben Miller

SMO – Shared Management Objects

Managment Studio uses SMO

SMO emits TSQL

Powershell is a scripting language

Objects – live in memory; have properties and methods.

How does SMO work? memory to engine and engine to memory; once the request is made there are no other requests – the data from that point in time is stored.

Server object contains the connection information

Default happenings –

  • get database engine type
  • compatability
  • database name

Database

Default fields – name

Interesting Fields –

  • ID (group)
  • Size (retrieved)
  • Owner (group)
  • PageVerify (group)
  • SpaceAvailable (retrieved)
  • CompatibilityLevel (retrieved)
  • CreateDate (group)
  • Many more…

Table

Default fields – schema, name, ID

Interesting Fields –

  • IndexSpaceUsed (retrieved)
  • IsSystemObject (group)
  • Rowcount (group)
  • Createdate (group)
  • LastModified (group)
  • Many more…

Columns

Default fields – Name, ID

Interesting Fields –

  • DataType (group)
  • Identity (group)
  • Nullable (group)
  • InPrimaryKey (group)
  • Others…

What now?

  • Iteration with ForEach
  • GetDefaultInitFields
  • SetDefaultInitFields
  • Use Profieler to continue to fish
  • Be specific if performance is key (one byte at a time)
  • Be aware of the pipline

Load assemblies

Old way –

[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO")

New way (sort of) –

Add-Type -AssemblyName "Microsoft.SqlServer.Smo"

Other new way –

Add-Type -AssemblyName "Microsoft.SqlServer.Smo, Version=11.0.0.0, Culture=Neutral,...."

Add-Type -AssemblyName "Microsoft.SqlServer.SmoExtended, Version=11.0.0.0, Culture=Neutral,...."

Powershell memory considerations – depending on what you are pulling back, and whether or not you have already retrieved the data, the time it takes for the command to be processed will vary.

Different ramifications with foreach.

SQL Saturday #223 – Beginning Powershell for DBAs 1.0 with Sean McCown

Powershell is EXCITING!!!

Write a single line of text to a file – It takes whole mess of VB Script to do what Powershell will do in ONE LINE.

“Book another holiday.” | out-file ./myfile.txt -append

List all the processes on the box – lots of VB code vs a single call to get-process in Powershell.

get-service | out-file ./myservices.txt

get-processes

psdrive – allows you to access the registry and sql server like you would any other drive.

Concatenate string – “My name is: $z” – will concatenate irregardless of data type of $z

$s = “localhost”

ping $s

$array = “server1”, “server2”, “server3”

$array

$array[2]

get-service | get-member (or use gm)

get-process | gm

get-service | format-table (FT) <list of columns, comma delimited>

FL – format list

get-service |%{$_.Kill()} – will kill all services and reboot your machine

invoke-sqlcmd

$server | %{invoke-sqlcmd -serverInstance $_ query “select * from ….”}

$server | %{invoke-sqlcmd -serverInstance $_  -Inputfile}

dir | %{$_.script()} -out-file ./mytables.txt

Carrier method

SQL Saturday #223 – Unraveling Tangled Code with Jen McCown

Organize

  • Format – when you change the formatting and then alter your procedure this will mess with your plan cache; SQL Refactor by Apex; SQLPrettifier
  • Comment –  Pseudocode; this should more of a description of what you are doing rather than the code itself

Breakdown

  • Structure plan of attack – sketch your query
  • Main query -> subquery ->subquery

Streamline

  • Obvious issues – extra rows, columns, unnecessary work, non-SARGable where clauses (cannot be indexed)
  • Streamline joins
  • Similar subqueries
  • Other – functions, etc.

Starting with something weird

For my first post I thought I would start things out with a bang and detail an issue that I recently encountered:

Location: “qstopsrt.cpp”:404
Expression: fFalse
SPID: 51
Process ID: 2908
Location: qxcntxt.cpp:1221
Expression: cref == 0
SPID: 51
Process ID: 2908
Msg 3624, Level 20, State 1, Line 1
A system assertion check has failed. Check the SQL Server error log for details. Typically, an assertion failure is caused by a software bug or data corruption. To check for database corruption, consider running DBCC CHECKDB. If you agreed to send dumps to Microsoft during setup, a mini dump will be sent to Microsoft. An update might be available from Microsoft in the latest Service Pack or in a QFE from Technical Support.
Msg 0, Level 20, State 0, Line 0
A severe error occurred on the current command. The results, if any, should be discarded.

In case you were wondering that CHECKDB was a waste of time (in this situation). Had never seen something like this before, but who doesn’t like a good challenge?

If you have seen this error, and tried to research it, you know there is not much out there. Most of what I found had to do with issues with the msdb or master databases. Those did not apply to my situation.

Anyways, on to the code:

SELECT TOP 50
[personid] as [personid],
[shoeid] as [shoeid],
[shoe_name] as [shoe_name],
DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) as [days_since_purchase],
FROM vw_yabba WITH (NOLOCK)
where [Paid_in_full] = '1'
ORDER BY [days_since_purchase] desc

SELECT TOP 50
[personid] as [personid],
[shoeid] as [shoeid],
[shoe_name] as [shoe_name],
[shoe_type] as [shoe_type],
[shoe_size] as [shoe_size],
DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) as [days_since_purchase],
dateadd(day,datediff(day,0,delivey_date),0) as [delivery_date],
CASE WHEN [Paid_in_full] = 1 THEN 'YES' ELSE 'NO' END as [Paid],
[purchase_date] as [purchase_date]
FROM vw_yabba WITH (NOLOCK)
WHERE 0 = 0
AND ((((1=1))))
AND ( ( ( (
([shoe_type] = 'Stiletto')
AND ([Paid_in_full] = '1')
AND ( (DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) = '23')
OR (DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) = '24')
OR (DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) = '25') )
AND ( ([shoe_rating] < '100') OR ([shoe_rating] IS NULL))) OR ( ([Paid_in_full] = '1') AND ( (DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) = '23') OR (DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) = '24') OR (DATEDIFF(day, CAST(purchase_date AS DATE), GETDATE()) = '25') ) AND ( ([shoe_rating] < '50') OR ([shoe_rating] IS NULL ) ) AND ( ([shoe_type] = 'mule') OR ([shoe_type] = 'athletic') OR ([shoe_type] = 'flat') ) ) ) ) ) ORDER BY [days_since_purchase] asc



Other useful background info:

  • Both queries produce the same error.
  • 3rd party application code - dynamically generated; cannot be changed.
  • Happend on a production reporting server.
  • View queries two tables with a union; tables are refreshed hourly.
  • Not all data is pulled from the tables - some fields are defined in the view.

Researched the error, but like I said - didn't find much that was of any use. Picked someone else's brain on the issue - he said there was an additional thing that could be added to the TOP 50 - WITH TIES. I changed the first query to a TOP 1 WITH TIES and got five records back. Kept working with both queries, trying different things. Eventually found that if I left the Paid_in_Full field out of the where statement the query would complete.

Examined the view - Paid_in_Full was defined as '1' in the view for one table and '0' for the other. Surmised from this that the implied data type could be the cause of the problem.

Solution that resolved the issue - modified the process that populates the tables to include this field (meaning the datatype would now be explicit).

Funny how something so simple could cause an issue so obscure.