Tuesday, June 12, 2018

My DevOp Project

I am into devOps for a week. The basic building block is to automate those systems. I have to say the PowerShell is a good tool to do this kind of work from a Win32 box. My first task is to call some services in a web site. I can find those services URL from the web page but need to go through some authentication processes.

The Invoke-WebRequest cmdlet + Chrome's network monitoring turns out to be the perfect solution to do this work.

First step is to understand how the web page performs.

I open the Chrome network monitoring tool by right click the web page and select "inspect". It shows the current DOM structure of displaying page. Click the "network" tab and select 'preserve log". The "preserve log" feature will keep all the logs even the page changes.

Now the target web page address can be pasted to Chrome and lots of log will be generated. I can see many page direction and form submission during this process (sorry, I cannot paste screenshot. 😉)

Step 2: Write down all the pages "network" tab shows after the initial web page is loaded. Please write down those web addresses. 

Now the simulation part starts. There are two variables needed for Invoke-WebRequest cmdlet:

  • SessionVariable: Browser maintains a session between client and server. So "-sessionVariable" parameter is a must. It will create a mutable variable to store session information between client and server, e.g. cookies. 
  • MaximumRedirection: PowerShell can automatically redirect to next page if you set the value to this variable > 0. I recommend to use 0 as initial test so you can know what is going on exactly.
Now let's the fun begin. 

Step 3: open PowerShell 
Step 4: run $r0 = invoke-webrequest -uri
-SessionVariable sessionVarialbe0 -MaximumRedirection 0

This will create the session variable "sessionVariable0". There is an error shows the maximum direction step is smaller. Please ignore this error. 

The cmdlet returns a $r0 variable, please check the $r0 content and you will find the web page address and redirection address from header field. Make sure the redirection address equals the one you get from Chrome network monitoring page (in step 2)

Step5: $r1 = inovke-webrequest -uri -webSession $sessionVariable0 -MaximumRedirection 0

Repeat the check performed on $r0.

if you cannot find the redirection address, please refer to "Form" variable in the $r1. You can invoke the address in the Form's action field and submit the fields in the HTTP request body part. For example:

$addressInForm = $r1.Forms[0].Action
$r2 = invoke-webrequest -uri $addressInForm  -SessionVariable sessionVarialbe0 -MaximumRedirection 0 -Method Post -UseDefaultCredentials -Body $r1.Forms[0].Fields

you might find the default browser opens, that's because PowerShell uses default browser to parse. The browser might show some error message, please ignore. 

You can repeat the above steps until you get the final destination. And the sessionVariable0 is the treasure from this journey, it contains all the information you need for future automation work. 

Happy hacking automation. ;)

Tuesday, July 29, 2014

SelfNote: Credit Risk Model

My first paper on Credit Risk model.

www.fonsrisksolutions.com/Documents/Using%20Default%20Rates%20FAJ.pdf is a good good paper. The unusual pattern "crisis-at-maturity" model is quite interesting to me.

Sunday, March 2, 2014

Setup Azure cloud on Windows Server

Service Bus is Azure Cloud solution, but it can be installed on your own hardware which must run Windows Server 2012.

Here are the links I think you need:
Installations and configuration instructions: http://msdn.microsoft.com/en-us/library/dn441394.aspx

the WebPI folder download folder is %LocalAppData%\Microsoft\Web Platform Installer\installers. You can get the WebPI installation file there.

the service bus connection string can be retrieved by using Get-SBClientConfiguration. Please make sure you import the certificate from the server to your local computer. Make sure use Get-SBAuthorizationRule to get the connection string with shared key.

Thursday, February 13, 2014

Reverse List Every K Node

Today I found an question and decide to use F# pattern match to give it a try.

The question is reverse a list every k node. For example, if the input list is [1; 2; 3; 4; 5; 6; 7; 8; 9; 10] and k = 3, the output will be [3; 2; 1; 6; 5; 4; 9; 8; 7; 10].

When I use pattern matching, I always like recursive implementation.

 let arr = [1..10]  
 let rec reverse arr i =   
   let reverse2 arr =   
     arr |> List.rev  
   let rec getHeadTail arr i =   
     match i with  
     | 0 -> ([], arr)  
     | _ ->   
       match arr with   
       | [] -> ([], [])  
       | h::t ->   
         let (head,tail) = getHeadTail t (i-1)  
         ([h]@head, tail)  
   let headArr, tail = getHeadTail arr i  
   match tail with  
   | [] -> reverse2 headArr  
   | _ -> reverse2 headArr @ reverse tail i  
 reverse arr 6  

Wednesday, February 12, 2014

F# Code Snippet Updated

I updated the F# code snippet addon for Visual Studio 2013. I will upload the source code to  once the source code is located. I really need to clean up the source code folder. There is no recompile is needed. The root cause for previous addon does not work on VS2013 is because this. The fix is simple, 

  1. download the code snippet vsix file
  2. rename it to .zip file because it is a zip file
  3. find the vsixmanifest file
  4. change InstallationTarget to

Sunday, November 17, 2013

Why F# Type Provider?

F# type provider is a very nice feature. I was asked several times why I need type provider? Just because of the intellisense? I believe there should be something else. In the current team, I found people "hate" type, they prefer to use C# object all the times. Now I realize why type is so important and why type can increase code quality.

I want to use a simple approval process as a sample. For my understanding, a type defines the meaning of a area of memory and certain operations which can be executed on this type.

The sample is simple. You can have something to be approved by lead, manager, and director. If a lead approves, it returns a manager type. If manager disapproves, the process will stop and director won't know it. After director's approval, this process stops. The type provider can have three types, director's approve method returns nothing while other type's approve method return higher level type. Lead's approve method returns manager, and manager's approve method returns director.

I'd bet people can say I can use a single type to do the same job. A general class can be used to create three instances. The director instance's approve method can return NULL or any special value to stop the approval process. It can solve this problem provides you know director's approval will stop this process beforehand. A runtime error is not far away if this information is not known beforehand. The type provider uses type to provide the information. After you invoke director's approve method, which returns NULL, user will know immediately that this process is terminated.

I would say in this case, type is a way to move runtime check/error to compile time. The attached is the demo project for this approval process.

Friday, November 8, 2013

Use F# to Write PowerShell Snapin & Cmdlet

When I was in Minneapolis, I was thinking to use the F# to write PowerShell snapin and cmdlet. I got the feeling that this can speed our development speed.

Because I use Visual Studio 2012 and it generates the .NET 4 binary. So I need to install PowerShell 3.0 in order to use .NET 4.0. You can use $psversiontable to check the CLR runtime version, make sure it is a number equal or greater than 4.

 namespace Log4NetPsSnapIn  
 open System  
 open System.Management.Automation  
 open System.ComponentModel  
 type Log4NetSnapIn() =  
   inherit PSSnapIn()  
   override this.Name with get() = "aa"  
   override this.Vendor with get() = "bb"  
   override this.Description with get() = "dd"  
 [<Cmdlet(VerbsCommunications.Write, "Hi")>]  
 type WriteHelp() =   
   inherit Cmdlet()  
   override this.ProcessRecord() =   

After compiling the above code, a DLL is generated. You have to use installutil.exe to add the snapin. The PowerShell snapin has Name = "aa", so you can use Add-PsSnapIn aa to load the DLL. You can also use installutil.exe /u to uninstall the snapin from your system. Make sure you open the cmd window or powershell window with administrator privilege.

  • If you are running 64-bit version machine, make sure you use installUtil.exe under C:\Windows\Microsoft.net\Framework64\. 
  • if your OS is 32-bit, you can use installUtil.exe under C:\Windows\Microsoft.net\Framework\

I like the way F# write PowerShell snapin. The code is concise and easier to understand.