Pages

Thursday, September 29, 2011

Bring intellisense when write F# query and other code

Ctrl + J is to bring the intellisense.
Ctrl +K, C is to comment out current selected code
Ctrl +K, U is to uncomment current selected code

I found Ctrl + J is very useful when writing F# queries. For example,

when you write something like:

let myQuery = query { for n in ABC do

Ctrl + J can help you find the keyword in query.

Wednesday, September 28, 2011

Trust type provider

This post is to help people get out "not trust" problem when he accidentally click "Disallow" on the security dialog.

I always click "enable" button to trust the type provider DLL. But today I accidentally click "disallow".. :-(

The rescue is here


  1. go to option under the tool
  2. find "F# tools"
  3. find "type providers"
  4. on the right side of panel, you can find the DLL listed and you can check the checkbox to trust it.

Tuesday, September 27, 2011

Pass in parameter with setter function



Not sure why the pass in parameter's property cannot be invoked. But I found a workaround.

[]
type A =  
    member this.Item
        with get(i:int) : float32 = failwith ".."
        and set (i:int) (v:float32) = failwith ".."

// this version does not work as c is not mutable
// let sample (a:A) (b:A) (c:A) =
//    c.[0] <- a.[0] + b.[0]

you have to use:

let sample (a:A) (b:A) (c:byref)= 
    c.[0] <- a.[0] + b.[0]

Sunday, September 25, 2011

Sample in Detail - How to write a regex type provider


How to write a your own type provider
Regular expression type provider is a good sample to show how to write a type provider from scratch. The sample code is publish can be viewed here.


Some feedback points the sample cannot be compiled using .fs file. I cannot get a repro. Here is the link to the new project file with .fs file inside.

Setup Environment
First setup the development environment:
1.            Create an F# solution with two projects: an F# console project and an F# library project.
a.            The console project is used for test
b.           The library project is the type provider project
2.            Set the console project as startup project
3.            In the console project, reference to F# library project
Because the library project is referenced and locked by the Visual Studio, so please close all the files from the test project and restart the solution before you want to compile your change to the library project. Other document suggests opened two visual studio instances. It is totally your choice.
Result
The following code is how this type provider is going to be invoked.
Figure 1 Regular expression type provider
type T = RegExProviderType<"(? < AreaCode > \d{3})-(? < PhoneNumber >\d{7}$)">
let reg = T()
let result = T.IsMatch("425-1232345")
let r = reg.Match("425-1232345").Group_AreaCode.Value //r = 425
      
·        RegExProviderType is the type provider name.
·        The string between angle brackets is the input to the type provider. This string is used to compute the properties and methods.
·        IsMatch is a RegExProviderType static method.
·        Match method on the last line is method take input string and return another generated type which contains property generated from the input string.
·        If the pattern is invalid according, the type provider should be smart enough to give a notification.
The method “IsMatch” and “Match” and property “Group_AreaCode” are generated from the input string.
Type Provider Template and Add Method and Property
The following table is a code snippet template to write a type provider.
Figure 2 Template for Type Provider
namespace

open System
open System.Linq.Expressions
open System.Reflection
open Microsoft.FSharp.Core.CompilerServices
open Microsoft.FSharp.Collections
open System.Collections.Generic
open Microsoft.FSharp.TypeProvider.Emit
open System.Text.RegularExpressions

type HelperClass() =    
type DebugType() 1=
    inherit System.Object()

[]
type public RegExTypeProvider() as this =
    inherit TypeProviderForNamespaces()
    let thisAssembly = Assembly.GetExecutingAssembly()
    let rootNamespace = "
    let baseType = typeof

    let regexType =
        let t = ProvidedTypeDefinition2(thisAssembly, rootNamespace, "", Some(baseType))
        t.DefineStaticParameters([ProvidedStaticParameter("pattern",typeof)]3,
                                    fun theTypeNameOfThisInstantiation args ->
                                        match args with
                                        | [| :? string as template |]4 -> 
                                            
                                            t                                                                                       
                                        | _ -> failwith "need generic definition"
                                        )

       //add a method or property
              
                                
        t
   
    do this.AddNamespace(rootNamespace, [regexType] )

[]
do()
1.       A DebugType is defined and this can help to identify the problem. Before the code is released, be sure to make it System.Object or other type. If the base type is set to object, it is easily confusing you when you see something like “expect type A while given System.Object”. You won’t be able to identify where to find the type A.
2.       Define the type provider type, this type is called “t”
3.       Define the input string. Its type is string and name is “pattern”.
4.       Take the value of input string as variable “template”. This is where we can access the pattern.
The type defined above is no use without any meaningful methods or property. Here is the way to add method:
Figure 3 Add Method
//add isMatch function
let parameters = [ ProvidedParameter("data", typeof) ]
let mi = ProvidedMethod("IsMatch", parameters, typeof)
mi.AddXmlDoc("Test if the input string matches the pattern")
mi.IsStaticMethod <- true
mi.InvokeCode <- fun args -> <@@ (Regex.IsMatch((%%args.[0]:string), template)) @@>
t.AddMember mi

The above code is very straightforward. A method named “IsMatch” is defined. It takes string as input and returns typeof.
The only tricky part is the %%args notation. If the method is a static method, the args is the parameter list. The first element in the args is the first parameter. If the method is a non-static method, the first argument is “this” pointer. The args is a list of “this” + parameter list. Please make sure the InvokeCode’s return type agrees to the type provided to ProvidedMethod.
If you want to add a property, there is something called ProvidedProperty type.
Figure 4 Add Property
//add RawRegex property
let pi = ProvidedProperty("RawRegex", typeof)
mi.AddXmlDoc("Return raw regex")
pi.IsStatic <- true
pi.GetterCode <- (fun args -> <@@ Regex(template) @@>)
t.AddMember pi


The code above defined a Regex type property called “RawRegex”. It return the Regex instance initialized from the variable “template”.
Because the regular expression type provider gets the pattern input string from outside. It is important to remember the #4 in the Figure 1 Template for Type Provider. The variable called “template” is the input string; this is a critical channel for a type provider to get information from outside world.
Embedded Type
At this point it is enough to write a simple type provider. We can easily finish 3 ½ line of code in Figure 1 Regular expression type provider code. The tricky part is how to get Group_AreaCode from the function call.
Because the Group_AreaCode is a property generated from the input string, this property must be the property from a generated type like RegExTypeProviderType. At this point, your intuition might already tell you that another ProvidedTypeDefinition is needed. Your intuition is correct.
Figure 5 MatchEx type
    //generate new type from pattern
    let matchType =
        let t = ProvidedTypeDefinition("MatchEx", Some(typeof))
       
        t

The new type is defined as “MatchEx” and based on Match. The reason we define this type is because we want to add group name to this type. The only place we can access input string is inside the “DefineStaticParameters” function. It makes sense we add the property over there.
Figure 6 Add property to the MatchEx type
//add properties for matchEx
let reg = Regex(template)
  for name in reg.GetGroupNames() do
    if name <> "0" then
       let propertyName = sprintf "Group_%s" name
       let pi = ProvidedProperty(propertyName, typeof)
       pi.GetterCode <- fun args -> <@@ ( (%%args.[0]:Match).Groups.[name] ) @@>
       matchType.AddMember(pi)


The code in Figure 5 Add property to the MatchEx type is to add property in the MatchEx type. Now we need to hook the MatchEx to its container type:
1.       Put Figure 4 MatchEx type inside the DefineStaticParameters.
2.       Add “t.AddMember(matchType)” at the end of the “fun theTypeNameOfThisInstantiation args” function. This line will link the embedded type.
Almost there! Next step is to change the Match function’s return type to this newly created type “MatchEx”.
Figure 7 MatchEx type with Match function
//add match function
let parameters = [ ProvidedParameter("data", typeof)]
let mi = ProvidedMethod("Match", parameters, matchType)
mi.AddXmlDoc("Match function")
mi.InvokeCode <- fun args -> <@@ (Regex(template).Match((%%args.[1]:string))) @@>
t.AddMember mi

If the highlighted part is how to change let a return type to be “MatchEx”. The matchType variable is defined in Figure 5 MatchEx type.

If you are thinking about using Activator.CreateInstance, you can use the above technique to make a function to return your embedded type. 

Sample in Detail - SQL Type Provider and SQL query

SQL TypeProvider samples are recently removed from the sample package. Those samples are trying to map SQL query to F# query. SQL query has been there for decades and large number of persons are using it every day.



SQL Statement (not case-sensitive)
F# query (case-sensitive)
C# query (case-sensitive)
1
select * from Student
where exists
(select * from CourseSelection
where CourseSelection.StudentID = 1)
let q =
    query {
        for i in db.Students do
        where (query { for c in db.CourseSelection do
                       exists (c.StudentID = 1) })
        select i }
var q = from n in db.Students
                    where (from m in db.CourseSelections
                           where m.StudentID == 1
                           select m).Any()
                    select n;
1a
select Student.Age, COUNT(*)
from Student
group by Student.Age
having student.Age > 1
let q = query {
        for n in db.Student do
        groupBy n.Age into g
        where (g.Key.HasValue && g.Key.Value > 1)
        select (g.Key, g.Count())
}

var q = from n in db.Students
                    where n.Age > 1
                    group n by n.Age into g
                    select new { g.Key, Count = g.Count() };
1b
select Student.Age, COUNT(*)
from Student
group by Student.Age
having COUNT(*) > 1
let q = query {
        for n in db.Student do
        groupBy n.Age into g
        where (g.Count()>1)
        select (g.Key, g.Count())
}

var q = from n in db.Students
                    group n by n.Age into g
                    where g.Count() > 1
                    select new { g.Key, Count = g.Count() };
1c
select Student.Age, COUNT(*), SUM(Student.Age) as total
from Student
group by Student.Age
let q = query {
        for n in db.Student do
        groupBy n.Age into g       
        let t = query { for n in g do sumByNullable n.Age }
        select (g.Key, g.Count(), t)
}
var q = from n in db.Students
                    group n by n.Age into g
                    where g.Count() > 1
                    select new { Age=g.Key, Sum=g.Sum(s=>s.Age), Count = g.Count() };
1d
select Student.Age, COUNT(*) as MyCount
from Student
group by Student.Age
having COUNT(*) > 1
order by COUNT(*) desc
let q = query {
        for n in db.Student do
        groupBy n.Age into g
        where (g.Count() > 1)
        sortByDescending ( g.Count() )
        select (g.Key, g.Count())       
}
var q = from n in db.Students
                    group n by n.Age into g
                    where g.Count()>1
                    orderby g.Count() descending
                    select new { Age = g.Key, Count=g.Count() };
2
select top 2 * from Student
where student.Name like '_a'
open System.Data.Linq.SqlClient;

let q = query {
    for n in db.Student do
    where (SqlMethods.Like( n.Name, "_a") )
    select n
    take 2  
    }
var q = (from n in db.Students                   
                    where SqlMethods.Like(n.Name, "_a")
                    select n).Take(2);
3
select * from Student
where student.Name like '[abc]%'
open System.Data.Linq.SqlClient;

let q = query {
    for n in db.Student do
    where (SqlMethods.Like( n.Name, "[abc]%") )
    select n   
    }
var q = from n in db.Students                   
                    where SqlMethods.Like(n.Name, "[abc]%")
                    select n;
4
select * from Student
where student.Name like '[^abc]%'
open System.Data.Linq.SqlClient;

let q = query {
    for n in db.Student do
    where (SqlMethods.Like( n.Name, "[^abc]%") )
    select n   
    }
var q = from n in db.Students                   
                    where SqlMethods.Like(n.Name, "[^abc]%")
                    select n;
5
select StudentID as ID from Student
where student.Name like '[^abc]%'
open System.Data.Linq.SqlClient;

let q = query {
    for n in db.Student do
    where (SqlMethods.Like( n.Name, "[^abc]%") )
    select n.StudentID   
    }
var q = from n in db.Students                   
                    where SqlMethods.Like(n.Name, "[^abc]%")
                    select n.StudentID;
6
select * from
Student left join CourseSelection
on Student.StudentID = CourseSelection.StudentID
let q2 = query {
        for n in db.Student do
        join (for e in db.StudentID -> n.StudentID = e.CourseID.Value)
        select n
    }

let q = query {
    for n in q2.DefaultIfEmpty() do
    select n   
    }
var q = from n in db.CourseSelections
                    join m in db.Students
                    on n.StudentID equals m.StudentID into g
                    from x in g
                    select x;
7
select * from
Student right join CourseSelection
on Student.StudentID = CourseSelection.StudentID
No match, you can reverse two tables.
No match, you can reverse two tables.
8
Select * from student
let q = query {
        for n in db.Student do
        select n
    }
var q = from n in db.Students
                    select n;
9
select count(*) from student
let q = query {
        for n in db.Student do       
        count
    }
var q = (from n in db.Students
                     select n).Count();
10
select COUNT(*) from
Student join CourseSelection
on Student.StudentID = CourseSelection.StudentID
let q = query {
        for n in db.Student do
        join (for e in db.CourseSelection -> n.StudentID = e.StudentID)
        count       
    }
var q = (from n in db.CourseSelections
                    join m in db.Students
                    on n.StudentID equals m.StudentID into g
                    from x in g
                    select x).Count();
11
select distinct StudentID from CourseSelection
let q = query {
        for n in db.Student do
        join (for e in db.CourseSelection -> n.StudentID = e.StudentID)
        distinct       
    }
var q = (from n in db.CourseSelections
                     select n).Distinct();
12
select distinct count(StudentID) from CourseSelection
let q = query {
        for n in db.Student do
        join (for e in db.CourseSelection -> n.StudentID = e.StudentID)
        distinct
        count      
    }
var q = (from n in db.CourseSelections
                     select n).Distinct().Count();
13
select * from Student
where Student.Name like '%A%'
(not case-sensitive)
let q = query {
        for n in db.Student do
        where (n.Name.Contains("a"))
        select n
    }
(not case-sensitive)
var q = from n in db.Students                   
                    where SqlMethods.Like(n.Name, "%A%")
                    select n;
14

select
* from Student
where Student.Name in ('A', 'B', 'C')

(not case-sensitive)
Open System.Linq  //please do not forget to add this line

let names = [|"a";"b";"c"|]
let q = query {
    for n in db.Student do
    if names.Contains (n.Name) then select n }
 (not case-sensitive)
var list = new List<string>(new string[] { "a", "b", "c" });
            var q = from n in db.Students
                     where list.Contains(n.Name)
                     select n;
15
select * from Student
where Student.Age between 2 and 5
let q = query {
        for n in db.Student do
        where (n.Age.Value >=1 && n.Age.Value <5)
        select n
    }
var q = from n in db.Students
                     where n.Age>=1 && n.Age<5
                     select n;
16
select * from Student
where Student.Age =1 OR Student.Age = 2
let q = query {
        for n in db.Student do
        where (n.Age.Value =1 || n.Age.Value =2)
        select n
    }
var q = from n in db.Students
                     where n.Age==1 || n.Age==2
                     select n;
17
select * from Student
where Student.Age =1 OR Student.Age = 2
order by Student.Name desc
let db = T.GetDataContext();
let q = query {
        for n in db.Student do
        where (n.Age.Value =1 || n.Age.Value =2)
        sortByNullableDescending n.Age
        select n
    }
var q = from n in db.Students
                    where n.Age == 1 || n.Age == 2
                    orderby n.Age descending
                    select n;
18
select top 2 student.Name from Student
where Student.Age =1 OR Student.Age = 2
order by Student.Name desc

let q = query {
        for n in db.Student do
        where ((n.Age.HasValue && n.Age.Value =21) || (n.Age.HasValue && n.Age.Value=22))
        select n.Name
        take 2
    }
var q = (from n in db.Students
                    where n.Age == 1 || n.Age == 2
                    orderby n.Name descending
                    select n).Take(2);
19
select * from Student
union
select * from lastStudent
let q = query {
        for n in db.Student do
        select (n.Name, n.Age)
    }

let q2 = query {
        for n in db.LastStudent do
        select (n.Name, n.Age)
        }

let q3 = q.Union (q2)
var q = (from n in db.Students
                     select n).Union(
                    (from n in db.LastStudents
                     select n));
20
select Student.Age, COUNT(*) from Student
group by Student.Age
Open System.Linq  //please do not forget to add this line

let q = query {
        for n in db.Student do
        groupBy n.Age into g
        select (g.Key, g.Count())
}

OR

let q = query {
        for n in db.Student do
        groupValBy n.Age n.Age into g
        select (g.Key, g.Count())
    }
var q = from n in db.Students
                    group n by n.Age into g
                    select new { Age=g.Key, Count=g.Count() }
21
select student.StudentID,
      case Student.Age
            when 1 then Student.Age * 100
            else student.Age
      end,
      student.Age
from Student
let q = query {
        for n in db.Student do
        select (if n.Age.HasValue && n.Age.Value=1  then (n.StudentID, System.Nullable(n.Age.Value*100), n.Age) else (n.StudentID, n.Age, n.Age))
    }
var q = from n in db.Students                   
                    select new { n.StudentID,  Age= n.Age==1 ? n.Age*100 : n.Age};
22
select * from Student
INTERSECT
select * from LastStudent
let q = query {
        for n in db.Student do
        select (n.Name, n.Age)
    }

let q2 = query {
        for n in db.LastStudent do
        select (n.Name, n.Age)
        }

let q3 = q. Intersect (q2)
var q = (from n in db.Students
                     select n).Intersect(
                    (from n in db.LastStudents
                     select n));
23
select * from Student, Course
let q = query {
        for n in db.Student do
        for m in db.Course do
        select (n, m)
}
(need more work to flatten the structure and grab out all fields)
var q = from n in db.Students
                    from m in db.Courses
                    select new { n, m };