Launched! Finding your next domain or identity through Exists

As we all know in the tech startup space. Find your next app’s name or social presence is a challenge sometimes.

We’ve all been there and trawled many sites and solutions to come up with that name. So a mate of mine (@MartinLaukkenen) and I have created a new web application called Exists.

Exists will check domains and social networks for that all crucial name, but it will also give you some suggestions through shortening, domain hacks and other secret methods.

Have a look at our announcement at;

Exists has launched!
http://discuss.exists.io/t/exists-has-launched/12

Try Exists at;

Exists
https://exists.io

Here are some screenshots of it in action;

screenshot-homepage

screenshot-results

screenshot-details

We look forward to your feedback.

Kristof Kowalski

WordPress Migration – Done

So I’ve had enough of my old host, GoDaddy.

Initially when I set the blog up some four years ago, it was simple and it did the job. Now the service seems to be going down hill fast when it comes to performance. Rendering a simple WordPress page in around 6 seconds is not my idea of decent service.

So here we are on a spiffy new Amazon EC2 instance.

One of the side effects of the move, for some reason the WordPress Export / Import utility stripped all the \’s from my posts. It all should be fixed but if some script doesn’t work for any reason, please let me know and I’ll have a look.

I’ve got some exciting projects in 2013 that I’m working on so I”m sure you’ll see more SharePoint, MongoDB and Ruby on Rails posts.

Kristof

PowerShell, MongoDB and Exporting GridFS Files

Still continuing along with the PowerShell, MongoDB and GridFS affair from my previous posts.

PowerShell, MongoDB and Searching GridFS
http://kowalski.ms/2012/07/25/powershell-mongodb-and-searching-gridfs/

PowerShell, MongoDB and GridFS File Uploads
http://kowalski.ms/2012/07/25/powershell-mongodb-and-gridfs-file-uploads/

We’re going to look at how you can quickly export files form MongoDB’s GridFS.

As mentioned in the previous articles, MongoDB comes with a mongofiles utility which will allow you to perform basic GET, PUT and LIST operations. Great for those quick sessions where you want to get a file or two, but what happens when it gets more involved? This is where this little script can come in handy.

The Script
As an example here, I’m searching for any files that have hit GridFS in the last 24 hours. Occasionally we’re interested in viewing the actual files and opening them up in their native applications from the file system.

Load your MongoDB C# Driver and Connect to MongoDB

#
# Load MongoDB C# Driver
#
Add-Type -Path "D:\Kristof\mongodb\MongoDB.Bson.dll"
Add-Type -Path "D:\Kristof\mongodb\MongoDB.Driver.dll"

#
# Connect to MongoDB
#
$db = [MongoDB.Driver.MongoDatabase]::Create('mongodb://server01/mydatabase?safe=true')

Export Your Files

# Export files from gridfs
$fileDir = "D:\Kristof\gridfsexport"
$files = $db.GridFS.FindAll()
foreach ($file in $files) {
if ($file.UploadDate -ge (Get-Date).AddDays(-1)) {
$fileName = $file.Name.ToString()
Write-Host "Exporting" $file.Name ":: Uploaded " $file.UploadDate
$fd = "$fileDir$fileName"
$fs = New-Object System.IO.FileStream($fd,[System.IO.FileMode]'Create')
$file.GridFS.Download($fs, $filename)
$fs.Close()
}
else {
#"It's before the date"
}
}

So there you have it. A nice, simple, perfectly formed script which will allow you to export the contents based on a property, in this case UploadDate. Extend this to our hearts content and hope it helps someone else.

Enjoy,

Kristof

SharePoint Large List Notifier

As any SharePoint Administrator might attest, good governance is always a challenge.

One issue that always seems to creep up on you is the size of your lists. Now we know in 2010 the list threshold was lifted to about 5000 items before you start seeing the fabled error;

This view cannot be displayed because it exceeds the list view threshold (5000 items) enforced by the administrator.

To view items, try selecting another view or creating a new view. If you do not have sufficient permissions to create views for this list, ask your administrator to modify the view so that it conforms to the list view threshold.

You can obviously increase the list threshold limit if you wish, which isn’t necessarily a great idea. You should really investigate why this is happening, in most cases applying and index to the list column would do the trick.

To help in governing large lists, here’s a simple script can iterate all your lists and then send an email for any lists that have gone over the threshold. This way you never skip a beat and you’re always on top of it.

The Script
The script will go through your lists and store any offending lists with their Name, Url and Items in a Data Table. Certain lists will always be massive so there is an exclusion array you can use so you dont get nagged by these serial offenders. Finally the script will email you if it finds any items of your desired threshold.

#
# SharePoint Large List Notifier
#

# Add PowerShell Snapin
Add-PSSnapin Microsoft.SharePoint.Powershell -ErrorAction SilentlyContinue

# Host site
$site = Get-SPSite http://portal.domain.local

# List Limit
$limit = 5000
# Exlude
$exclude = @("TaxonomyHiddenList",
    "User Information List",
    "Accepted List 01",
    "Accepted List 02"
    )

# Create Data Table
$table = New-Object System.Data.DataTable "Large Lists"
# Create Columns
$col1 = New-Object system.Data.DataColumn Title,([string])
$col2 = New-Object system.Data.DataColumn Url,([string])
$col3 = New-Object system.Data.DataColumn Items,([int])
# Add Columns
#Add the Columns
$table.Columns.Add($col1)
$table.Columns.Add($col2)
$table.Columns.Add($col3)

foreach ($web in $site.AllWebs   ) {
    foreach ($list in $web.Lists  ) {
        if ($list.ItemCount -gt $limit) {
                # Uncomment if you want to see the results on the console
                # Write-Host "Title:" -ForegroundColor Black -BackgroundColor Cyan -NoNewline
                # Write-Host " " -NoNewline
                # Write-Host $list.Title -ForegroundColor White
                # Write-Host "  URL: " -ForegroundColor DarkCyan -NoNewline
                # Write-Host $list.DefaultViewUrl -ForegroundColor Gray
                # Write-Host "Items: " -ForegroundColor DarkCyan -NoNewline
                # Write-Host $list.ItemCount -ForegroundColor Gray

                # Exclude sites that can have more items
                if ($exclude -contains $list.Title) {
                    # Ignore
                }
                else {
                    # Add into table
                    $row = $table.NewRow()
                    $row.Title = $list.Title
                    $row.Url = $list.DefaultViewUrl
                    $row.Items = $list.ItemCount
                    $table.Rows.Add($row)
                }

            }

            else {
                # Ignore
            }

    }
}

if ($table.Rows.Count -eq 0) {
    # Do nothing
}
else {
    $tableSites = $table | Format-Table -AutoSize | Out-String
    # Send email containing large lists
    # SMTP Server
    $smtpServer = "smtp.domain.local"
    # Net Mail Object
    $msg = New-Object Net.Mail.MailMessage
    # SMTP Server Object
    $smtp = New-Object Net.Mail.SmtpClient($smtpServer)
    # Email
    $msg.From = "Large.Lists@domain.local"
    $msg.ReplyTo = "Large.Lists@domain.local"
    $msg.To.Add("Recipient.One@domain.local")
    $msg.To.Add("Recipient.Two@domain.local")
    $msg.CC.Add("Recipient.Three@domain.local")
    $msg.Subject = "[WARNING] The following list/s are above $limit items."
    $msg.Body =
"Greetings,

The following lists have more items then the recommended limit;

$tableSites
Regards,`n

--
Large List Notifier
"
    # Send Message
    $smtp.Send($msg)
}

# Clear and Dispose Table
#$table
$table.Clear()
$table.Dispose()

Hopefully you’ll be in a better place now to manage those pesky large lists we always strive to maintain.

Enjoy,

Kristof

PowerShell, MongoDB and Searching GridFS

Following on from my previous post;

PowerShell, MongoDB and GridFS File Uploads
http://kowalski.ms/2012/07/25/powershell-mongodb-and-gridfs-file-uploads/

You’re storing your files on GridFS but how can you search for anything in there? Normally you’d need to jump into your MongoDB Shell and perform something like;

use mydatabase
db.fs.files.find({});

{ "_id" : ObjectId("4fe855bd355b900fd8ddeecc"), "chunkSize" : 262144, "contentTy
pe" : "text/xml", "filename" : "xmlfile-016593.xml", "l
ength" : NumberLong(1740), "md5" : "5cb2502a0a5f7de34bdb03f66ddc16eb", "uploadDa
te" : ISODate("2012-06-25T12:12:45.336Z") }

Since we’re working in the PowerShell world, that’s not really much use to us, so lets go ahead and create something that will allow us to search for terms inside of GridFS.

I’ve written a Search-MongoGridFS function that will use the GridFS.Opentext() method to open the file and then we match a term using the Select-String cmdlet. The output will list the matching files that will contain the desired term.

The Function

#
# Search for strings in MongoDB's GridFS function
#

function Search-MongoGridFS
{
    Param (
        [CmdletBinding()]

        [Parameter(
            Mandatory=$true,
            HelpMessage='Path to the MongoDB C# Driver? MongoDB.Bson.dll and MongoDB.Driver.dll')]
        [string]$Driver,

        [Parameter(
            Mandatory=$true,
            HelpMessage='What server do you want to search?')]
            [Alias('s', 'svr')]
        [string]$Server,

        [Parameter(
            Mandatory=$true,
            HelpMessage='What database do you want to search?')]
        [string]$Database,

        [parameter(
            Mandatory=$false,
            HelpMessage='What term do you want to search?')]
            [Alias('t')]
        [string]$Term
    )

    Begin {
        Clear-Host
        # Load MongoDB C# Driver
        Add-Type -Path "$path\MongoDB.Bson.dll"
        Add-Type -Path "$path\MongoDB.Driver.dll"

        # Connect to MongoDB Server and Database
        #$db = [MongoDB.Driver.MongoDatabase]::Create('mongodb://localhost/komplex?safe=true;slaveok=true')
        $connection = 'mongodb://' + $server + '/' + $database + '?safe=true'
        $mongo = [MongoDB.Driver.MongoDatabase]::Create($connection)
        # localhost, komplex
    }

    Process {
        [array]$found = @()
        $total = 0
        $files = $mongo.GridFS.FindAll()
        foreach ($file in $files) {
            $total++
            $perc = ($total / $files.Count()) * 100
            Write-Progress -activity "Searching for $term..." -status "Searching..." -PercentComplete $perc
            $read = $file.GridFS.OpenText($file.Name)
            $string = Select-String -InputObject $($read.ReadToEnd()) -SimpleMatch $term
            if ($string -ne $null) {
                #$file.Name
                $found += $file.Name
            }
            else {
                # ignore
            }
            $read.Dispose()
        }
    }

    End {
        $found
        Write-Host
        Write-Host "Found " -ForegroundColor DarkCyan -NoNewline
        Write-Host $found.Count -ForegroundColor Cyan -NoNewline
        Write-Host " files of " -ForegroundColor DarkCyan -NoNewline
        Write-Host $files.Count() -ForegroundColor Cyan -NoNewline
        Write-Host " containing the term " -ForegroundColor DarkCyan -NoNewline
        Write-Host $term -ForegroundColor White -NoNewline
        Write-Host "." -ForegroundColor DarkCyan
        Write-Host
        $found.Clear()
    }
}

To perform a search all you need to do is supply the path to your C# driver, the server to search, the database to search and finally the term. The syntax is as follows;

Search-MongoGridFS -driver D:\Kristof\mongodb -server server01 -database mydatabase-term Awesome

Hopefully this will allow you to search your MongoDB GridFS stores and save you some headaches. Obviously you can expand this script to search files and other properties super easy.

Enjoy,

Kristof

PowerShell, MongoDB and GridFS File Uploads

As part of one of the startups I’m working on our web front-end architecture is based on Microsoft’s ASP.NET MVC 3 framework, but our back-end storage is MongoDB. We’re obviously sitting outside of the traditional Open Source camp by using Microsoft technology as our web framework, even though ASP.NET is now open sourced. You don’t get all the goodness of the rich set of community snipits that other more traditional frameworks (Rails, Django) provide.

With that in part we’ve had to write a lot of helper scripts to make our lives just that little bit easier. One area that definately needs some help is MongoDB’s GridFS with PowerShell.

A little background to GridFS first.

GridFS
“GridFS is a storage specification for large objects in MongoDB. It works by splitting large object into small chunks, usually 256k in size. Each chunk is stored as a separate document in a chunks collection. Metadata about the file, including the filename, content type, and any optional information needed by the developer, is stored as a document in a files collection.”

Reference:
http://www.mongodb.org/display/DOCS/GridFS

With MongoDB you get a simple tool called mongofiles which allows you you to perform basic commands such as, GET, PUT and LIST. This is great if you want this simplicity else you need to roll your own solution. In our case we wanted to reuse some of our PowerShell scripts to create some richer tools.

With that in mind here is the first post of several with working with PowerShell, MongoDB and GridFS.

Load your MongoDB C# Driver
Grab it from the MongoDB CSharp Language Center

#
# Load MongoDB C# Driver
#
Add-Type -Path "D:\Kristof\mongodb\MongoDB.Bson.dll"
Add-Type -Path "D:\Kristof\mongodb\MongoDB.Driver.dll"

Find Your Files
As part of being a good citizen you need to ensure you have the correct Content Types when you upload to GridFS. I generally do a simple Get-Unique on a directory, use the output to generate a hash table of my Content Types.

#
# Find File Types
#
$files = Get-ChildItem -Path "D:\Kristof\Uploads"
Get-ChildItem "D:\Kristof\Uploads" | Select-Object Extension | Sort-Object Extension | Get-Unique -AsString

Create Content Type Hash Table
This is our hash table from the above output. Create an entry for each file extension it lists.

#
# Create Has Table of Content Types
#
$fileType = @{
  "avi" = "video/x-msvideo";
  "exe" = "application/octet-stream";
  "html" = "text/html";
  "jpg" = "image/jpeg";
  "js" = "application/x-javascript";
  "mp3" = "audio/mpeg";
  "pdf" = "application/pdf";
  "png" = "image/png";
  "txt" = "text/plain";
  "xml" = "application/xml";
  "zip" = "application/zip"
}

Upload Files to GridFS
Now that we have our hash table sorted we’re ready for the upload process to start.

#
# Connect to MongoDB
#
$db = [MongoDB.Driver.MongoDatabase]::Create('mongodb://server01/mydatabase?safe=true')

#
# Upload Files to GridFS
#
$files = Get-ChildItem -Path "D:\Kristof\Uploads"
$number = $files.Count
$x = 0
foreach ($file in $files) {
  $x++
  $fileName = $file.FullName
  $fs = new-object System.IO.FileStream($fileName,[System.IO.FileMode]'Open')
  $fileOptions = new-object MongoDB.Driver.GridFS.MongoGridFSCreateOptions
  $fileOptions.ContentType = $fileType.Item($($file.Extension).Substring(1,3))

  Write-Progress -activity "Copying file $file ..." -status "Copying $x of $number"

  $db.GridFS.Upload($fs, $file.Name, $fileOptions) | Out-Null
  $fs.Close()
}

Simple as that. GridFS is a wonderfully simple technology to use and when used in the correct method it’s a real winner.

Enjoy,

Kristof

Working with MongoDB and PowerShell

For one of our side projects, we’ve chosen to use a Microsoft’s ASP.NET MVC 3 framework for our Web Front-Ends and 10gen’s MongoDB situated on top of Ubuntu Server. For the domain we’re working in this has suited us nicely in this project.

As the more Ops focused one in the team I’ve been wanting to leverage my PowerShell foo with MongoDB. It’s always great to step out of your comfort zone and try something new. I wanted to get basic CRUD (create, read, update and delete) functionality working and here are the fruits of my labour.

I assume you’ve already played around with MongoDB or some other NoSQL database. If not, jump over to the the MongoDB site and have see for yourself what you may/may not be missing;

MongoDB
http://www.mongodb.org

So lets get started. Firstly download the latest MongoDB C# drivers from;

mongodb / mongo-csharp-driver
https://github.com/mongodb/mongo-csharp-driver/downloads

Extract then out to a directory. I like to keep all my bins together, so my path will be;

c:\mongodb\bin

Now we’ll need to load the Mongo C# assemblies

PowerShell v1.0 – Use Reflection

[Reflection.Assembly]::LoadFile("c:\mongodb\bin\MongoDB.Bson.dll")
[Reflection.Assembly]::LoadFile("c:\mongodb\bin\MongoDB.Driver.dll")

PowerShell v2.0 – Use Add-Type

Add-Type -Path "c:\mongodb\bin\MongoDB.Bson.dll"
Add-Type -Path "c:\mongodb\bin\MongoDB.Driver.dll"

Now that we have our assemblies load we should be able to leverage the MongoDB.Bson and MongoDB.Driver namespaces. I’m going to assume you already have a MongoDB instance running somewhere.

The next step is to create a db connection to our Mongo instance;

$db = [MongoDB.Driver.MongoDatabase]::Create('mongodb://localhost/awesome?safe=true;slaveok=true')

Reference: Connections
http://www.mongodb.org/display/DOCS/Connections

I’m running a local instance on my machine for testing purposes, hence why it’s localhost, no authentication and no other servers replica/config servers have been specified.

If all goes well, you should be able to query your Mongo server as follows;

[MongoDB.Driver.MongoServer]::GetAllServers().BuildInfo

Your output should be similar to the following;

Bits          : 64
GitVersion    : 927beb54fb5ec38ccd0ca5fa712564a0696c9003
SysInfo       : windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack='Service Pack 1') BOOST_LIB_VERSION=1_49
Version       : 2.1.2.0
VersionString : 2.1.2

A massive thanks to thanks to Justin for originally blazing the trail with PS and Mongo integration;

Using MongoDB in PowerShell
http://www.justaprogrammer.net/2011/05/14/using-mongodb-in-powershell

We’ll need to define a collection called to store our documents in;

$collection = $db['Users']

Now lets create a basic BSON document to store in our Users collection of Awesome DB;

[MongoDB.Bson.BsonDocument] $doc = @{
    "_id"= [MongoDB.Bson.ObjectId]::GenerateNewId();
    "FirstName"= "Kristof";
    "LastName"= "Kowalski";
    "PhoneNumbers"= [MongoDB.Bson.BsonDocument] @{
        'Home'= '+44-208-1000-1000';
        'Mobile'= '+44-7841-100-100';
    };
};

Lets go ahead and insert the document in the Users collection;

$collection.Insert($doc)

Reference: Inserting
http://www.mongodb.org/display/DOCS/Inserting

To ensure the document was saved you can simply run;

$collection.FindAll()

Reference: Querying
http://www.mongodb.org/display/DOCS/Querying

Your output should be similar to the folowing;

Name                                                                                        Value                                                                                     
----                                                                                        -----                                                                                     
_id                                                                                         4ff44a1e355b9013ecc198f7                                                                  
PhoneNumbers                                                                                {Mobile=+44-7841-100-100, Home=+44-208-1000-1000}                                         
FirstName                                                                                   Kristof                                                                                   
LastName                                                                                    Kowalski                                                                                  

So we’ve got the C and R out of the way, now we want to try the U and the D. Lets work with the above example and add an email address. Since it’s NoSQL, we dont really care about schema changes too much. Well, you do but thats for another post, but this is an uber simple example of the beauty of NoSQL databases.

Lets use the the above document and add an email address field to our user using the $set modifier;

$update = @{'$set' = @{'email'= 'kristof@kowalski.ms'}}
$query = @{"_id"= $doc['_id']}
$collection.Update([MongoDB.Driver.QueryDocument]$query, [MongoDb.Driver.UpdateDocument]$update)

Reference: Updating
http://www.mongodb.org/display/DOCS/Updating

If you perform an $collection.FindAll() you should notice that our user now has an email address of type String associated with it. Simples.

Lets finally perform a delete of our document;

$collection.Remove([MongoDB.Driver.QueryDocument]$query)

If you want to play around with querying, updating and deleting on our collection here’s quick script that will generate some random users and their assocaited phone numbers. There’s no better way to learn then actually getting down and dirty with it.

# Insert 1000 users into the Users collection
$x = 0
while ($x -le 1000)
{
    $firstName = @( "John", "Mike", "Andrew", "Bob", "Stuart", "Aaron", "Abbott", "Abel", "Acelin", "Adon", "Rafi", "Randal", "Randall" ) | Get-Random
    $lastName = @( "Smith", "Jones", "Andrews", "Scully", "Kowalski" ) | Get-Random
    $homeNumber = "+44-208-" + (Get-Random -Count 1 -InputObject (1000..9999)).ToString() + "-" + (Get-Random -Count 1 -InputObject (1000..9999)).ToString()
    $mobileNumber = "+44-7841-" + (Get-Random -Count 1 -InputObject (100..999)).ToString() + "-" + (Get-Random -Count 1 -InputObject (100..999)).ToString()

    [MongoDB.Bson.BsonDocument] $doc = @{
    "_id"= [MongoDB.Bson.ObjectId]::GenerateNewId()
    "FirstName"= "$firstName"
    "LastName"= "$lastName"
    "PhoneNumbers"= [MongoDB.Bson.BsonDocument] @{
        'Home'= $mobileNumber
        'Mobile'= $homeNumber
                }
            }
    Write-Host "Inserting document $x"
    $collUsers.Insert($doc) | Out-Null
    $x++
}

I’ll be posting some more details about using advanced queries and GridFS in the next round of posts, so stay tuned. Give MongoDB a whirl and I’m sure you’ll be pleasantly surprised.

Enjoy,

Kristof

Foursquare, PowerShell and IIS Monitoring Fun

As part of our daily Ops lives, monitoring is massive part of making sure our systems are performing in a stable and performant fashion. Now lets face it, SCOM and Nagios or [Insert Favourite Monitoring Solution Here], aren’t anything to get excited about.

I’ve been a foursquare (4sq) user for some time now and love the service. Now wouldn’t it be cool, if one of my servers gets and error it would notify me somehow over 4sq?

Well, fear not people, it’s possible.

We have a particularly annoying problem on IIS that happens once in a blue moon that affects our XSLTs on our SharePoint farms, very annoying. WAS Error 5011 (scream) happens every couple of days on a random web front-end and we need to recycle the App Pool to alleviate this corruption. Thanks XLV Web Parts and heap corruptions! *sigh*

Now, when we receive this error my little app called PingBot will checkin to one of the world’s more notorious jails and I know there’s an issue. When the problem is fixed, after an hour he’ll/she’ll/it will check back in our Development Corner venue automatically.

Example Checkin

So without further ado here’s what you’ll need;

  1. A foursquare login – http://www.foursquare.com
  2. A foursquare OAuth2 App Client Id – https://developer.foursquare.com
  3. An OAuth2 AccessToken

Firstly want to say a massive thank you to Trevor Sullivan for initially writing the PowerShell function to get our OAuth2 access Token. If you have no idea what OAuth2 is, jump across to his site for a nice intro;

http://trevorsullivan.net/2012/05/18/powershell-getting-an-access-token-from-instagram-oauth-2-0

Firstly we’ll need to get our Access Token. For this I’ve altered Trevor’s function to suite foursquare’s OAuth2 service provider. This will give you the Access Token that will allows you to query the foursquare API.

Get your Access Token

function Get-oAuth2AccessToken {
    [CmdletBinding()]
    param (
          [Parameter(Mandatory = $true)]   [string] $AuthUrl
        , [Parameter(Mandatory = $true)]   [string] $ClientId
        , [Parameter(Mandatory = $true)]   [string] $RedirectUri
        , [int] $SleepInterval = 2
        
    )
    
    # Build the scope of permissions
    if (!$Scope) {
        $ScopeString = 'basic'
    }
    foreach ($item in $Scope) {
        $ScopeString += $item + '+'
    }
    $ScopeString = $ScopeString.TrimEnd('+')

    # Build the request URL from a template    
    $RequestUrl = '{0}?client_id={1}&redirect_uri={2}&response_type=token' `
                -f $AuthUrl, $ClientId, $RedirectUri, $ScopeString
    Write-Debug -Message ('Request URL is: {0}' -f $RequestUrl)
    
    # Create the Internet Explorer object and navigate to the constructed authorization URL
    $IE = New-Object -ComObject InternetExplorer.Application
    $IE.Navigate($RequestUrl)
    $IE.Visible = $true
    
    # Sleep the script for $X seconds until callback URL has been reached
    # NOTE: If user cancels authorization, this condition will not be satisifed
    while ($IE.LocationUrl -notmatch 'access_token=') {
        Write-Debug -Message ('Sleeping {0} seconds for access URL' -f $SleepInterval)
        Start-Sleep -Seconds $SleepInterval
    }
    
    # Parse the access token from the callback URL and exit Internet Explorer
    Write-Debug -Message ('Callback URL is: {0}' -f $IE.LocationUrl)
    [Void]($IE.LocationUrl -match '=([\w\.]+)')
    $AccessToken = $Matches[1]
    $IE.Quit()

    # Write the access token to the pipeline inside of a HashTable (in case we want to return other properties later)
    Write-Debug -Message ('Access token is: {0}' -f $AccessToken)
    Write-Output -InputObject @{ AccessToken = $AccessToken }
}

#Enable debug messages to be sent to console
$DebugPreference = 'continue'

# The base authorization URL from the service provider
$AuthUrl     = 'https://foursquare.com/oauth2/authorize'
# Your registered foursquare application Client ID
$ClientId    = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
# The callback URL configured on your application. Must be valid!
$RedirectUri = 'http://www.yourdomain.com'                                

# Get Access Token
Get-oAuth2AccessToken -AuthUrl $AuthUrl -ClientId $ClientId -RedirectUri $RedirectUri

Since PowerShell v3 is right around the corner I really wanted to get to know the new cmdlets so I’ve written PingBot in a way that it will leverage v3s new Invoke-RestMethod, Invoke-WebRequest and the new JSON serializer. Notice how many lines the v2 code is to the v3? Egads!

Foursquare PingBot

#
# Ping Bot Checker
#

# Your application's foursquare UserId
$userId = "XXXXXXXX"
# Your application's foursquare Access Token
$accessToken = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
# Your aplication's API version, basedon current date
$version = (Get-Date -Format yyyyMMdd).ToString()

# Venues
# Your Home Venue, etc 5500 - Dev Corner
$ourHome = "4fa3f133e4b0e7038a278824"

# Error Venues
# Alcatraz Island
$alcatraz = "4451c80ef964a520a5321fe3"
# San Quintin State Prison
$sanQuintin = "4a348baef964a520639c1fe3"
# Hỏa Lò (Hoa Lo Prison or "Hanoi Hilton")
$hanoiHilton = "4bb022def964a5202c393ce3"
# Robben Island
$robbenIs = "4bb70b6046d4a593ce0ec7c0"
# Long Bay Jail
$longBay = "4bf4bfd594af2d7fcd023b72"

# Get random error jail venue - Jails of the world
$random = @( "$alcatraz", "$sanQuintin", "$hanoiHilton", "$robbenIs", "$longBay" )  | Get-Random

# Servers to check error on
$computers = @("SERVER01", "SERVER02", "SERVER03", "SERVER04")

foreach ($computer in $computers)
{
    # Get-EventLog didn't return the Message correctly
    $event = Get-EventLog -LogName System -Newest 1 -Source "WAS" -ComputerName $computer -After (Get-Date).AddMinutes(-15) | Select -First 1    
    if ($event.Id -eq 5011) {
            # Get error message as Shout        
            [string]$id = $event.Id.ToString()
            [string]$message = $event.Message.Substring(0,167)
            $shout = "Crikey! $computer has $id! $message ..."

            if ((Get-Host).Version.Major -eq 3) {                
                # Get venue name
                $urlVenues = "https://api.foursquare.com/v2/venues/" + $random + "?" + "oauth_token=" + $accessToken + "&v=" +$version
                $venue = Invoke-RestMethod $urlVenues -Method Get                
                [string]$venueName = $venue.response.venue.name

                # Checkin to venue
                Write-Host "Error detected! Checking into $venueName!"
                $urlCheckins = "https://api.foursquare.com/v2/checkins/add?venueId=" + $random + "&shout=" + $shout + "&oauth_token=" + $accessToken + "&v=" + $version
                Invoke-WebRequest -Uri $urlCheckins -Method Post
            }
            else {                
                # Get venue name
                $urlVenues = "https://api.foursquare.com/v2/venues/" + $random + "?" + "oauth_token=" + $accessToken + "&v=" +$version                
                $wc = new-object net.WebClient
                $obj = $wc.downloadData($urlVenues)
                $s = 1::ascii.getString($obj)

                # Convert JSON to PowerShell
                [Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions") | Out-Null
                $c = New-Object System.Web.Script.Serialization.JavaScriptSerializer
                $cVenue = $c.DeserializeObject( $s )
                [string]$venueName = $cVenue.response.venue.name
                                
                # Checkin to venue
                Write-Host "Error detected! Checking into $venueName!";
                $urlCheckins = "https://api.foursquare.com/v2/checkins/add?venueId=" + $random + "&shout=" + $shout + "&oauth_token=" + $accessToken + "&v=" + $version
                $command = '{"jsonrpc": "2.0", "method": "JSONRPC.Version", "id": 1}'
                $bytes = [System.Text.Encoding]::ASCII.GetBytes($command)
                $web = [System.Net.WebRequest]::Create($urlCheckins)
                $web.Method = "POST"
                $web.ContentLength = $bytes.Length
                $web.ContentType = "application/x-www-form-urlencoded"
                $stream = $web.GetRequestStream()
                $stream.Write($bytes,0,$bytes.Length)
                $stream.close()

                # If you want to see checkin output
                #$reader = New-Object System.IO.Streamreader -ArgumentList $web.GetResponse().GetResponseStream()
                #$reader.ReadToEnd()
                #$reader.Close()                
            }            
    }       

    else {
            if ((Get-Host).Version.Major -eq 3) {                
                # No error found, but check to see if home checkin is needed. Checkin to home if error appeared more then 1 hour ago.
                # Get checkin createdAt time
                
                $urlUserId = "https://api.foursquare.com/v2/users/" + $userId + "?" + "oauth_token=" + $accessToken + "&v=" + $version                
                $cUserId = Invoke-RestMethod $urlUserId -Method Get                
                [string]$items = $cUserId.response.user.checkins.items
                
                $items = $cUserId.response.user.checkins.items
                foreach ($item in $items) {
                    $epochDate = $item.createdAt
                    }                
                $createdAt = [timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1970').AddSeconds($epochDate))
                
                # Checkin to home again
                if ( $createdAt -le (Get-Date).AddHours(-1)  ) {                    
                    #
                    Write-Host "Already Home"
                }
                else {                    
                    # Checkin to Home      
                    Write-Host "Checking to home!"              
                    $shoutHome = "Home! Sweet Home!"
                    $urlCheckins = "https://api.foursquare.com/v2/checkins/add?venueId=" + $ourHome + "&shout=" + $shoutHome + "&oauth_token=" + $accessToken + "&v=" + $version
                    Invoke-WebRequest -Uri $urlCheckins -Method Post | Out-Null
                }
             
             }                
            else {
                # No error found, but check to see if home checkin is needed. Checkin to home if error appeared more then 1 hour ago.
                # Get checkin createdAt time
                $urlUserId = "https://api.foursquare.com/v2/users/" + $userId + "?" + "oauth_token=" + $accessToken + "&v=" + $version                
                $wc = new-object net.WebClient
                $obj = $wc.downloadData($urlUserId)
                $s = 1::ascii.getString($obj)

                # Convert JSON to PowerShell
                [Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions") | Out-Null
                $c = New-Object System.Web.Script.Serialization.JavaScriptSerializer
                $cUserId = $c.DeserializeObject( $s )
                
                $items = $cUserId.response.user.checkins.items
                foreach ($item in $items) {
                    $epochDate = $item.createdAt
                    }                
                $createdAt = [timezone]::CurrentTimeZone.ToLocalTime(([datetime]'1/1/1970').AddSeconds($epochDate))
                
                # Checkin to home again
                if ( $createdAt -le (Get-Date).AddHours(-1)  ) {                    
                    #
                    Write-Host "Already Home"
                }
                else {
                    # Checkin to Home               
                    Write-Host "Checking to home!"     
                    $shoutHome = "Home! Sweet Home!"
                    $urlCheckins = "https://api.foursquare.com/v2/checkins/add?venueId=" + $ourHome + "&shout=" + $shoutHome + "&oauth_token=" + $accessToken + "&v=" + $version
                    $command = '{"jsonrpc": "2.0", "method": "JSONRPC.Version", "id": 1}'
                    $bytes = [System.Text.Encoding]::ASCII.GetBytes($command)
                    $web = [System.Net.WebRequest]::Create($urlCheckins)
                    $web.Method = "POST"
                    $web.ContentLength = $bytes.Length
                    $web.ContentType = "application/x-www-form-urlencoded"
                    #$stream = $web.GetRequestStream()
                    #$stream.Write($bytes,0,$bytes.Length)
                    #$stream.close()
                }
            }                    
        }        
}

I have this task running every 15 minutes on an admin box, but a word of warning, you need to be wary that you dont abuse the API rate limits set by fousquare as they’ll block your id. Check their documentation at;

Rate Limits
https://developer.foursquare.com/overview/ratelimits

Who says you can’t have a bit of fun with your monitoring regime aye?

Enjoy!

Kristof

Customising Your PowerShell Prompt

It’s been an age since I’ve written anything and I have a massive amount of posts which I still want to get out.

Like with any Windows admin you’ll be working with PowerShell on a daily basis, so why not tweak your “prompt” so it suits you. To modify your prompt firstly you’ll need to understand how PowerShell Profiles work, so have a quick gander over here before you start;

Windows PowerShell Profiles
http://msdn.microsoft.com/en-us/library/windows/desktop/bb613488(v=vs.85).aspx

For my prompt I like to inject a little bit of pizazz and extra information, after all a little bit of information goes a long way as they say. This prompt is pretty much based on my many years of using Linux and have a custom BASH prompt.

I like to see the current time, who I’m logged in as on what machine (ideal of PSRemoting) and last of all, what directory I am in and whether I’m an Administrator on a given machine or not.

The following code;

function prompt
{        
    
    # Set Window Title
    $host.UI.RawUI.WindowTitle = "$ENV:USERNAME@$ENV:COMPUTERNAME - $(Get-Location)"
    
    # Set Prompt
    Write-Host (Get-Date -Format G) -NoNewline -ForegroundColor Red
    Write-Host " :: " -NoNewline -ForegroundColor DarkGray
    Write-Host "$ENV:USERNAME@$ENV:COMPUTERNAME" -NoNewline -ForegroundColor Yellow
    Write-Host " :: " -NoNewline -ForegroundColor DarkGray
    Write-Host $(get-location) -ForegroundColor Green
    
    # Check for Administrator elevation
    $wid=[System.Security.Principal.WindowsIdentity]::GetCurrent()
    $prp=new-object System.Security.Principal.WindowsPrincipal($wid)
    $adm=[System.Security.Principal.WindowsBuiltInRole]::Administrator
    $IsAdmin=$prp.IsInRole($adm)
    if ($IsAdmin) {        
        Write-Host "(admin) #" -NoNewline -ForegroundColor Gray
        return " "
    }
    else {        
        Write-Host ">" -NoNewline -ForegroundColor Gray
        return " "
    }
 }

Should give you something like this. Notice the first PowerShell prompt has no “(admin) #” as you’ve not elevated yourself with Administrator privileges, as opposed to the second one. A neat little UAC admin indicator.

Non Admin

Admin

One powerful feature is leveraging the jobs infrastructure. If you have any long running scripts you can set it to runasjob with the Invoke-Command. The prompt will show you any jobs with a Running state and then disappear once they have finished. Here’s the altered jobs version of the above prompt for PowerShell v3

function prompt
{        
    
    # Set Window Title
    $host.UI.RawUI.WindowTitle = "$ENV:USERNAME@$ENV:COMPUTERNAME - $(Get-Location)"
    
    # Set Prompt
    Write-Host (Get-Date -Format G) -NoNewline -ForegroundColor Red
    Write-Host " :: " -NoNewline -ForegroundColor DarkGray
    Write-Host "$ENV:USERNAME@$ENV:COMPUTERNAME" -NoNewline -ForegroundColor Yellow
    Write-Host " :: " -NoNewline -ForegroundColor DarkGray
    Write-Host $(get-location) -ForegroundColor Green

    # Check Running Jobs
    $jobs = Get-Job -State Running
    $jobsCount = $jobs.Count
    
    # Check for Administrator elevation
    $wid=[System.Security.Principal.WindowsIdentity]::GetCurrent()
    $prp=new-object System.Security.Principal.WindowsPrincipal($wid)
    $adm=[System.Security.Principal.WindowsBuiltInRole]::Administrator
    $IsAdmin=$prp.IsInRole($adm)
    if ($IsAdmin) {
        if ($jobsCount -eq $null) {
            Write-Host "(admin) #" -NoNewline -ForegroundColor Gray
            return " "
        }
        else {
            Write-Host "(admin) jobs:" $jobsCount -NoNewline -ForegroundColor Gray
            Write-Host "#" -NoNewline -ForegroundColor Gray
            return " "
        }        
    }
    else {                
        if ($jobsCount -eq $null) {
            Write-Host ">" -NoNewline -ForegroundColor Gray
            return " "
        }
        else {
            Write-Host "jobs:" $jobsCount  -NoNewline -ForegroundColor Gray
            Write-Host ">" -NoNewline -ForegroundColor Gray
            return " "
        }        
    }
 }

Enjoy.

Kristof

SharePoint 2010 Export Managed Metadata Terms from the Term Store

As part of our SharePoint 2010 deployment here, we make extensive use of the new Managed Metadata Service Application. To ensure we have concise and correct terms we have Term Store Managers that check on a weekly basis the validity of these terms. Some of the tests they check is duplicaiton or terms, correct spelling and just general term lifecycle itself.

I’ve checked around and there are plenty of scripts around that instruct you to upload and create terms through PowerShell, but we needed a quick way to export our current terms and find out who created the term and when. So here’s a litte script you can schedule to run each week.

One thing to check for is the name of the Term Store you bind, which shoukd be listed via the Get-SPTaxonomySession cmdlet.

# Add SharePoint PowerShell Snapin
Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

 # File and Directory Location
$dirLocation = "C:\Temp\Terms\"
$date = get-date -Format yyyyMMdd
New-Item ($dirLocation + $date) -Type Directory | Out-Null
$file = New-Object System.IO.StreamWriter(($dirLocation + $date) + "\Terms.csv")

# Connect to site with MMS service connection
#$taxonomySite = Get-SPSite "http://site"
$taxonomySite = Get-SPSite -Limit 1
 
# Connect to Term Store in the Managed Metadata Service Application
$taxonomySession = Get-SPTaxonomySession -site $taxonomySite
$taxonomyTermStore =  $taxonomySession.TermStores | Select Name
$termStore = $taxonomySession.TermStores[$taxonomyTermStore.Name]

# Ampersands are stored as full width ampersands within the MMS database.
[Byte[]] $amp = 0xEF,0xBC,0x86

# CSV headers
$file.Writeline("Term Name,Id,Owner,CreatedDate,LastModifiedDate")

# Term counter
$i = 0

foreach ($group in $termStore.Groups) {
	
	foreach ($termSet in $group.TermSets) {
		
		foreach ($term in $termSet.GetAllTerms()) {
			[Byte[]] $amp = 0xEF,0xBC,0x86;
			$file.Writeline("""" + $term.Name.Replace([System.Text.Encoding]::UTF8.GetString($amp), "&") + """" + "," + $term.Id + "," + $term.Owner + "," + $term.CreatedDate + "," + $term.LastModifiedDate);
			$i++
			Write-Host -ForegroundColor Cyan  "# Exporting TermSet: " -NoNewline
			Write-Host -ForegroundColor White $termSet.Name -NoNewline
			Write-Host -ForegroundColor Cyan  " Term: " -NoNewline
			Write-Host -ForegroundColor White $term.Name -NoNewline
			Write-Host -ForegroundColor Green " - Done"			
			}			
		}				
	}
$file.Flush()
$file.Close()

Write-Host
Write-Host -ForegroundColor Cyan  "# Exported " -NoNewline
Write-Host -ForegroundColor Green  $i -NoNewline
Write-Host -ForegroundColor Cyan  " terms"

With the above script that will export all your Terms with the Term Name, Id, Term Owner, Creation Date and Last Modificaiton Date. Our Term Store Managers like to review the terms on a weekly basis so all that needs to be run later is the following couple of lines per each Term Set;

# If you want see what terms have changed since last export, ie 7 days
$dateToCompare = (Get-date).AddDays(-7)
Import-Csv '.\Terms.csv' | Where-Object {$_.LastModifiedDate -gt $dateToCompare} | Sort-Object {$_.LastModifiedDate} -Descending

I’ll alter the above script later to iterate through all the Term Set csv files to then export the changes in a further update. For now, enjoy.

Cheers,

Kristof Kowalski | kristof@kowalski.ms