Pages

Saturday, November 3, 2018

SelfNote: Stanford NLP

I create this page as the master page for using F# on Stanford NLP.

Monday, October 29, 2018

F# Stanford NLP is running

After some configuration, I can successfully run the first NLP project with F#. Special thanks to Sergey's post! The post is very informative. His solution is based on the F# interactive while I prefer to use the project-based solution.

Sergey points out that one of the common problems to setup is the path problem. His claim is so true. I had stuck in this problem for days. Here is the process I followed.


  • Open Visual Studio 2017 and create an F# console application. 
    • I tried .net core app; it does not work as the IKVM has the dependency on the .NET framework
  • compile the F# console application and remember the debug folder location
  • Open NuGet and retrieve Stanford NLP CoreNLP. The current version is 3.9.1
    • Current Stanford NLP is 3.9.2. I suggest you download 3.9.1 version
  • download the Standard NLP 3.9.1 zip file
  • unzip the 3.9.1 file to the F# console app debug folder
  • go the unzipped folder and find the model JAR file

  • download WINRAR to unzip the JAR file to a folder, this folder should contain a folder called "EDU"
  • copy the "EDU" folder up to debug folder, so the structure in the "DEBUG" folder is like the following.
  

The F# file I was using is listed below. Set the "EDU" folder to the debug folder can save you from setting the CurrentDirectory. 


1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
// Learn more about F# at http://fsharp.org
// See the 'F# Tutorial' project for more help.

open System
open System.IO
open java.util
open java.io
open edu.stanford.nlp.pipeline

[<EntryPoint>]
let main argv = 
    let text = "Kosgi Santosh sent an email to Stanford University. He didn't get a reply.";

    // Annotation pipeline configuration
    let props = Properties()
    props.setProperty("annotators","tokenize, ssplit, pos, lemma, ner, parse, dcoref") |> ignore
    props.setProperty("ner.useSUTime","0") |> ignore

    let pipeline = StanfordCoreNLP(props)

    // Annotation
    let annotation = Annotation(text)
    pipeline.annotate(annotation)

    // Result - Pretty Print
    let stream = new ByteArrayOutputStream()
    pipeline.prettyPrint(annotation, new PrintWriter(stream))
    printfn "%O" <| stream.toString()
    stream.close()

    printfn "%A" argv
    0 // return an integer exit code

Executing the NLP program seems taking a lot of memory. My program uses 2G memory and takes a while to show the result. Hopefully, your computer is faster enough. :)

Thursday, October 25, 2018

F# Enum usage II

I want to take a quick note on the F# enum usage again. The supporting of the space and unique character in F# language and editor is a great feature can make your development work much more comfortable. I am in the Web API front these days.

One of the requirement is to provide options to end users. If an application only takes "excellent choice", "good option", "ok choice", and "bad and never go there" as options, I'd like to offload these values check to the type system instead of handling the error in my code.

If I can declare the enum like the following

type enum MyEnum =
    ``excellent choice`` = 0
    | ``good option`` = 1
    ....

After declaring the enum-as-string attribute on the attribute [JsonConverter(typeof(StringEnumConverter))], the output and input validation is solved in one shoot.

Monday, September 24, 2018

F# Enum's Usage

I have a web service project and I found my team constantly needs to convert enum to a string. The only conversion is adding space. It wastes lots of time and involves in reflection which slows down the run time performance.

Enum type is one of the favorite types in F#. C#'s enum type does not have the ability to define a value with space. Having an enum value with space is very important because this feature can save me lots of time to output enum value as string.

For example, if the enum value can be "Post Release", the ToString function can output a nice string and won't have to use attribute and reflection to do the job. C# does have the ability to have space in enum but it will need to use reflect and emit to generate the type. If you can use F#, the problem can be easily solved. Please check the following F# code:

namespace ClassLibrary1

type public EnumEng =
    Registration = 0
    | ``Under Review``=1
    | Approval = 2
    | Release = 3
    | ``Post Release`` = 4

type public EnumChn =
    注册 = 0
    | 审批 = 1
    | 批准 = 2
    | 发布 = 3
    | 发布后 = 4

type public EnumIndex =
    Reg = 0
    | Review = 1
    | Approval = 2
    | Release = 3
    | PostRelase = 4


module Test =
    let a = EnumEng.``Under Review``
    let b = EnumChn.审批

In the C# side, the intellisense won't display the enum value if it contains space. However, it will be displayed in debug mode. You can execute the following code and stop at the end of the function.

        static void Main(string[] args)
        {
            //get all string from enum0
            var strs = Enumerable.Range(0, 5)
                                 .Select(n => (ClassLibrary1.EnumEng)n)
                                 .Select(n => n.ToString())
                                 .ToList();

            // get all string from enum2
            var i = Enumerable.Range(0, 5)
                              .Select(n => (ClassLibrary1.EnumChn)n)
                              .Select(n => n.ToString())
                              .ToList();

            // parse string to enum
            var v = strs.Select(n => Enum.Parse(n))
                        .ToList();

            var x = ClassLibrary1.EnumIndex.PostRelase;

            var str = (ClassLibrary1.EnumChn)x;
        }

Both # and C# support non-English variable name, it will provide a way to localize the output as well. From the sample above, the EnumIndex is the type can used in C#/F# code. Once the value needs to be output as string, it can be convert to EnumEnglish (English string) or EnumChinese(Chinese string).



Is that convenient?

Thursday, September 13, 2018

SelfNote: WebAssembly With Blazor & .NET Core Upgrade

After set up the HTML5/Typescript roadmap for my group, I started to move my interest to other UI/visualization technology. With the Blazor is in the starting phase, I feel this is an opportunity to get real-time web rendering and .net core in one shoot. The get start part in Blazor is very helpful. Only one small bug for the project creation if you upgrade .NET core.

From PowerShell window, you can find the .Net core version by using "dotnet --info". The Blazor service generates the global.json file, which sits beside the solution file. This file is required to load the .Net core. The default value in the file is 2.1.300. 

{
  "sdk": {
    "version": "2.1.300"
  }
}
My computer is new and I directly installed 2.1.402 version. Now I got the error complains about "cannot import package". After change that version to 402, I can now manually add those created projects (Blazor Server and Client) to the solution.  

Sunday, September 9, 2018

A Tech Manager's View on Full Stack Developer

As a hiring manager, I was asked by agent about full stack developer. I am not a big fan of so-called full stack developers. 

No one can have unlimited memory and time to be expert in all area. I am more interested in knowing what the depth one can go. Those average stuff can be searched from internet. Those experience does not demonstrate the critical thinking skills. Those solution, with high probability, introduces high maintenance cost down the road.

Being deep in one area requires perseverance and talent, those attributes will provide a foundation to long lasting solution. 

Being a technology manager, maintaining different technology stack has a huge cost factor and segment team into small silos. The team effort will be lost. The tech plan and road map is missing. 

I have read some job description that requires tens of unrelated technology. This kind of position really makes me thinking how effective the tech management is and what the team looks like.

Friday, August 31, 2018

Technology from a Tech Manager's View - Measure Productivity

It is always an intriguing topic about software development productivity. There are some attempts to measure productivity. From those numbers, there are some conclusions and team chaos and management challenge are knocking at the door.


First of all, I do not think this productivity is measurable. If a computer program can understand how much effort to solve a problem, this program will be smart enough to solve the problem. Then we don’t need software developers at all.

The worst thing about this measurement is let team members know about it. The assumption is measurement will give insight into the problem and eventually leads to a solution. Not mention the trust between employee and management is broken, the assumption itself has a fatal error. You can measure stone's size and weight and those numbers won’t change. However, the subject of the measurement is human who will change the behavior if s/he knows someone is watching.

Naturally the subject (human) changes behavior to make the number better. Will this change really improve the productivity? Most likely not. The first thing human subject can do is to game the system. Will this effort result in high quality software? No. When you know you are not trusted, are you willing to contribute more?

Einstein's noble prize paper is only a few pages. Essentially the attempt to measure is put equivalence between software development activities and operational labor work.