Sunday, December 30, 2007

JavaNCSS Hackystat Sensor

I just started to look at how to write a Hackystat sensor for the JavaNCSS results. A couple of interesting thoughts and issues came up.

is Resource enough?
what is a Resource in Hackystat? well, according to the Rest API Specification, a Resource is a "A URI indicating the resource (file, website, etc.) on which this sensor data was collected." Hmm.. Okay that seems fair enough. But, the problem I'm faced with is that JavaNCSS produces package, object, and method level metrics. Thus, it would be impossible to capture and utilize all of JavaNCSS's metrics.

for example, here is some sample data:
<package>
<name>org.hackystat.sensor.ant.checkstyle</name>
<classes>2</classes>
<functions>7</functions>
<ncss>129</ncss>
<javadocs>6</javadocs>
<javadoc_lines>32</javadoc_lines>
<single_comment_lines>19</single_comment_lines>
<multi_comment_lines>17</multi_comment_lines>
</package>

<object>
<name>org.hackystat.sensor.ant.checkstyle.CheckstyleSensor</name>
<ncss>67</ncss>
<functions>4</functions>
<classes>0</classes>
<javadocs>4</javadocs>
</object>

<function>
<name>org.hackystat.sensor.ant.checkstyle.CheckstyleSensor.CheckstyleSensor()</name>
<ncss>2</ncss>
<ccn>1</ccn>
<javadocs>1</javadocs>
</function>

so you see that package has some data that object does not (single_comment_lines and multi_comment_lines). and function has ccn (Cyclomatic Complexity Number). To satisfy Hackystat's requirement I'd just have to use the object metrics, which means missing out on the single_comment_lines, multi_comment_lines, and ccn. That seems bad to me.

Mapping inner classes to files
Currently, when the tool results do not provide the file path (ie provides a class name org.hackystat.sensors.ant.checkstyle.CheckstyleSensor), we do a matching to match the class name with a file. I'm not sure how that works exactly, but I do wonder if it works correctly for inner classes. Basically, would org.hackystat.sensor.ant.checkstyle.CheckstyleSensor.Temp (where Temp is an inner Class) map correctly to c:\java\svn\hackystat-ant-sensors\src\org\hackystat\sensor\ant\checkstyle\CheckstyleSensor.java?
I'm not sure it would.

JavaNCSS bug
Investigating that inner class issue, I found that JavaNCSS does not capture inner class metrics correctly. it seems that JavaNCSS doesn't report separate metrics for inner classes. i created two temporary inner classes within the CheckstyleSensor to test out whether JavaNCSS would handle it correctly. unfortunately, it seems the only metrics i get out of JavaNCSS about those inner classes are rolled up in existing elements, meaning i don't get a separate object element for the inner classes. instead, i get this:

<object>
<name>org.hackystat.sensor.ant.checkstyle.CheckstyleSensor</name>
<ncss>75</ncss>
<functions>4</functions>
<classes>2</classes>
<javadocs>4</javadocs>
</object>

the classes=2 is supposed to represent the two inner classes that i added. i suppose, one could argue that the inner classes belong to CheckstyleSensor and that should be okay to contain the metrics within the CheckstyleSensor object. But, wouldn't it be better if we could distinguish the metrics between CheckstyleSensor and its inner classes. anyway... i didn't particularly like that about JavaNCSS.


Anyway, a Hackystat JavaNCSS should be relatively easy to implement. i probably could have finished it faster than writing this blog. but, i wanted to hash out some issues that i was thinking about. in fact, i think i should bring up the Resources and mapping to file questions to the hackystat-dev mailing list.

Friday, December 28, 2007

hackystat and my outboard brain

i just had an interesting idea that branches off of what wired has been talking about a lot; an outboard brain. see Welcome to a Science Journalist's Outboard Brain and help me help you.

basically, what i want is a hackystat and outboard brain combined. this is interesting because, i can learn a lot from people's outboard brain. i can learn a lot more from "watching" them work.

imagine for example, that you knew everything that philip thought was cool and you knew what kind of things philip was working on and the processes he used to work on them. if you had that then, you'd be pretty aware of philip's intentions, his direction, and you'd learn a whole lot from that.

this is a cool concept. but, i haven't had enough time to think deeper about this.

Friday, December 21, 2007

Finding what you are passionate about

Here is a good post that kind of sums my perspective on life and work: Finding the right work (what you're passionate about) is like discovering your own soul in this universe.

"When you love what you do then it is no longer work, it becomes a way of life."


I love what I do. Our job as a company is to find the people that share a similar passion and help our current employees find and embrace their own drive. In my opinion that is what separates ordinary companies from spectacular ones. Our job as a company is to embrace passion. When you think about it, that's not too hard.

Wednesday, December 19, 2007

interview with austen ito

i have been working with austen ito for a few years now. austen is a product of our collaborative software development laboratory research lab. austen is a up and coming hacker. and in my opinion austen is one of the top software engineers for his experience in the state.

here is how this interview works. the questions that i ask are highlighted in bold. austen's responses are in italics.


Austen, you have mentioned that you knew that you were going to major in Information and Computer Science. What triggered that decision? Talk about some of the things that you did in your childhood that peaked your interests.


I'm probably in the minority when i say that I knew I wanted to major in Computer Science since I was a freshman in high school. I may not know have known what Computer Science was at the time, but I knew that software was what I wanted to do. In high school I found my way to internet relay chatrooms (IRC) using a program called mIRC (http://www.mirc.com/). The neat thing about mIRC was that it allowed you to write scripts to customize your chat client. It was there I found the basics of for loops, boolean logic, timers, conditional statements, all of which could be visually seen in the chat client.

Now that I think about it, it was actually the first place where I was introduced to Open Source Software. I used to go to http://www.mircscripts.org/archive/scripts to find scripts written by other people. I would try out scripts that others have written and hacked it up a bit to do brand new things. There were also script developer communities and teams that were exciting, scripters that I could talk to on different channels, and I even had QA testers (awesome online friends) that would test out scripts that I wrote.

I did this off and on for a couple of years. Playing on the internet in chatrooms was one of the best things I could've done because it sparked my interest in software. I didn't realize what I was doing at the time. I was just having fun creating new things and getting feedback from my friends online.


Wow, that is cool. Haha, I have no idea what you are talking about though. I had a totally different path to software engineering. Sounds like you did have a lot of fun back then learning and hacking. Are you having that much fun now that you are in the real world? If so, what makes it fun. If not, what do you have to change to get back to the good old days?

Hrm tough question. I think the definition of fun comes down to "What type of problem am I trying to solve?" When I was younger, I didn't have a real problem in mind. I would just hack away at something because I enjoyed it. It was rewarding when I got something to work, but that was the end of it. At my current job, I am working on a fairly interesting problem that has the potential to be something very useful for your customers. In the context of work hacking, developing software for a customer's needs is fun and new experience for me.

I also hack in my free time, and again I have a different type of fun. The problems that the Hackystat team is trying to solve is something that, in my opinion, can make a huge difference. I find that being a part of a software team that has the possibly of revolutionizing the way people think about software engineering is awesome. The ideal situation for me would be to combine both types of fun into work
hacking.

Now I can answer your question. I think that I am having more fun now because I am doing something useful.


I'm glad that things are going well. Moving on to our next question. Lets dream a little. Forget about work stuff and Hackystat; erase those from your mind. Now think about a project that would be the ultimate useful, fun, revolutionary, cool. In other words, if I gave you 10 million dollars to do a software project, what would it be. What is the ultimate project for you?

I thought about this question for the entire weekend... And I couldn't think of a specific project in mind. Maybe my inexperience and lack of project diversity is showing. Something that did come to mind was the business model of Joel Spolsky:

The Best Working Conditions + Best Programmers + Best Software = Profit

I would love to be be a part of a company that follows his business model. 10 million dollars would make it possible to create a startup that could make profit by depending on the best developers working on things they are passionate about.


I totally agree with the Spolsky business model. Shifting gears a little; you mentioned inexperience. So, I'll end on this last question. What things do you think you need or want to work (ie skills, projects, masters) on to grow that experience?

I think that things I need to work on in order to grow my experience is to continue working on interesting software projects. As a new developer, learning skills such as new technologies and languages are great. You can't become awesome if you don't know about those things. But more importantly, I need to work on projects that teach me other things such working in high-performing high-quality teams, learning from and mentoring others, and being a part of something great. My hope and goal is to move onto a project that will expand my abilities as a developer by working with awesome people. Working on side projects will definitely help out with my experience problem. Hacking is a lifestyle, so I need to continue to be disciplined and keep
practicing in order to grow my software experiences.

[end interview]

my comments
austen is one of the few people that i know that knew right away that software is what he wanted to do. i wish i knew what i wanted to do at an early age. i think that says something about his motivation.

austen mentioned that he isn't that experienced. i agree and disagree with his viewpoint of himself. austen is probably a far better hacker than i. but, as i pointed out to him in one of discussions "innovation doesn't really care what your code looks like". i think there needs to be a balance of both science and coding. coding horror just put this out: no one cares what your code looks like.

in any case, i think austen is doing all the right things. he is reading books, practicing his craft (coding), learning to lead, learning, etc, etc. he is on the right track; can you other readers out there keep up with him?

more clover work

i've made some progress on getting clover working and the clover hackystat sensor. i feel compelled to talk a little more about clover. here are a few of my other clover posts; implementing the clover sensor and metrics clouds.

so, the clover tool looks pretty cool. however, i'm not sure how much better it is than something like emma. one thing is certain, clover has a lot nice reports. here is an example:



that is pretty nice. i suppose that is the benefits of a commercial tool. however, i found bad sides of a commercial tool. for example, the licensing costs is ridiculous:



for some reason the forums were pretty empty. i guess it could be because of the recent buy out of Cenqua. i want a bigger community behind it.



or maybe its the cost of the tool itself. anyway, i'm not sure why no one is posting. all i know is that i would be crazy to want to buy this tool at full price.

i do think clover is a good tool and i would use it.

google reader feature requests

some ideas for google reader features

  • i want google reader to only show the subscriptions that have unread postings. this would make it a lot easier to do a "gu" shortcut and jump to the subscriptions that need reading. i want this feature. i would hack on this feature.
  • a better notification system for new posts. i like to stay on top of my readings so that i don't have hundreds of postings to sort through. i do this fairly quickly. i mark the postings that i don't care to read as read. and i leave the interesting ones unread. over the course of the week i go back and read the ones that are interesting. when using reader like that, for some reason i want to know when new postings are available so i can quickly sort through them. i find that to be a lot more efficient than waiting till the end of the week.
  • send the tags along with the posting. i wish there was away to send the tags or labels of the postings through the subscriptions. that would help me determine a lot faster whether i care about the posting or not. one could imagine a more verbose organization of subscriptions.
  • wouldn't it be interesting to see a tag or word cloud of the postings that you star or share. now that would be interesting.

    those were some random ideas that i made up in 5 minutes. its rather interesting that i can do that so quickly. that indicates to me that google reader is a good application. its good because i see the potential. its good because i want to use it. and its good that i want to use its future features. if you think about the application that you are working on for work or as a side project and you can't generate a list like that, then i would argue that it isn't that interesting. or worse yet, your application lacks a direction.
  • Tuesday, December 18, 2007

    i got 2 people to use google reader today

    inspired by google readers new friend shared feature, i set out to get people to use google reader and share stuff. i got two friends to try it out. yay!

    how come i'm all excited? i'll tell you why.

    i've been trying to get people to share (via reader) what they thought were interesting for a long time. i thought that writing things in my blog was good enough; for example, twitter, sharing, and hackystat goals. but for some reason, i didn't get through to people (people that actually read my blog). it took some manual coaching and the fact that the google reader had a lot of hipe up this feature helped a lot too. i think the key thing was the manual coaching that made the difference.

    i really think sharing things like this sets a foundation of getting to know someone in a distributed collaboration. meaning, that i know what kinds of things that the person likes and thinks is interesting. i learn a lot from people just by looking at what they share. and this really ties in nicely with what the wired folks are doing with their help me help you idea. as a side effect, i'm exposed to so much more than i ever will find out by my self. like i've been saying its like a 10 second vignette of knowledge.

    lastly, i made this up the other day. i sent my non-google-reader friend some interesting links the other day and told him you owe me 5 interesting links next time. i call this link debt. i share you share kind of thing. i think that is a really cool idea.

    anyway, i think sharing is a really good concept. google reader has its flaws but its still pretty useful. i enjoy learning about others through this sharing concept.

    Sunday, December 16, 2007

    geosearch


    energyshortage's use of google earth is an example of something similar to what i had in mind in things i want from google.

    imagine for a second that you searched "energy shortage" in google. up popped up a google earth representation of all the instances of energy shortage "events" around the world. instantly you would see how expansive the concept was. if there was only energy shortage events in america and there 50, then not so bad right? but what if there was 50,000 only in america, then that would probably be bad. geosearch is a way to understand the context of what you are searching for without having to read text to figure it out.

    wow.. that is cool. see, i have good ideas!

    Saturday, December 15, 2007

    think about things other than code

    bill gates (http://blog.wired.com/business/2007/12/bill-gates-the.html):

    I don't think you can overemphasise the importance of having a good background in maths and science. If you look at the most interesting things that have emerged in the last decade - whether it is cool things like portable music devices and video games or more practical things like smart phones and medical technology - they all come from the realm of science and engineering.


    writing code isn't everything. math and science equals innovation too. i really believe that.

    help me help you

    i was reading some of the Wired blogs like Epicenter and Wired Science (actually two of my favorite blog sources), when i stumbled upon a "help me help you" idea.

    basically, it all started with Alexis' attempt to provide transparency into his thinking. he decided to do that with sharing his "shared blogs", his twitters, and facebook page with his readers. he challenges his readers to help shape what he writes about by learning more about the things that he thinks about. (read about it here; My Outboard Brain Is Your Chance to Shape News).

    that was an interesting idea. he is basically saying "help me help you". he wants his readers to understand who he is by viewing his interests. and he wants his readers to help him understand what they are interested in by hopefully creating a community of collaborating readers.

    the other wired bloggers followed his lead:
    There's a serious point to this. Journalists are no longer the gatekeepers of the news. Instead, we're hubs -- conduits -- and we'll succeed to the extend that we're able to facilitate the flow of information. We think tools like Facebook, Twitter and LinkedIn can help us do that better.

    And yes, this is one big experiment. We don't have all the answers.

    this is cool. it is cool because they are trying to use social networking technology to create something other than just social networking. this is something i've been trying to describe in my posts. my point has been that social networking is pretty much meaningless entertainment, unless it is applied to "something bigger". in this specific case, i think it is cool because they are taking "sharing to a new level".

    Thursday, December 13, 2007

    re: Encouraging people to contribute knowledge

    the official google blog put this article out today, encourage people to contribute knowledge

    the blog is very interesting and it has some of the things about sharing that i've been talking about for a while:

    There are millions of people who possess useful knowledge that they would love to share, and there are billions of people who can benefit from it. We believe that many do not share that knowledge today simply because it is not easy enough to do that. The challenge posed to us by Larry, Sergey and Eric was to find a way to help people share their knowledge. This is our main goal.


    the article quickly turns into an advertisement for something similar to wikipedia; and i didn't want to highlight that. thats too bad because i thought they were on track when they said, ""knol", which stands for a unit of knowledge", which is similar to my thought about 10 second vignette of knowledge. in that sense i think they might be falling short of their mission. wikipedia, and i guess neither will knol, doesn't really capture or encourage sharing. i suppose, it will initially, as people generate content. after a while you have a critical mass of things and then it will just sit there. anyway, i think the google blog identifies a very worthwhile goal, but i'm not sure their solution really attacks the problem as much as it should.

    anyway, i was really excited for a little while after reading the first couple of paragraphs, then said to my self, "oh my they are copying wikipedia". in that sense i guess it was a disappointment.

    don't be afraid to speak up

    (i have been doing a lot of self reflection lately.)

    one of the things that i keep on noticing is that some people are quiet, some people say things, and some people say interesting and thought provoking things. i notice that people that speak up, are more recognized for their thoughts. duh... the opposite is true, people that don't say anything aren't asked for their thoughts.

    i think i speak up and i think i say interesting things. well, that is my goal. i don't care if i'm right or wrong. its the learning experience of being able to share my thoughts and being able to communicate it effectively that drives my out outspokenness.

    the other day at work, a consultant pointed out that i ask a lot of very hard questions. that made me feel good; because that demonstrated my level of understanding and being able to push that understanding to new levels. after a little while of feeling good, i reflected...

    i began to think about other people. i began to think about who (taking a cross section of all sorts of people that i know) also asks interesting and thought provoking questions. i began to categorize these people. i noticed some interesting things.

  • people that ask more of the interesting things are able to think outside of the box
  • people who don't really speak up, probably wouldn't be able to write their thoughts out either.
  • some people will dwell on what they know really good; asking sometimes inappropriately detailed and irrelevant questions.
  • people that ask good questions are able to draw their knowledge from varying sources; making connections that other people cannot.

    those are just some of my observations about other people. i speak up a lot of times, because i just want to know more about things. my bottom line is that i want to learn, and i want to be able to practice my communication skills, and my thought process. i get better at that sort of things every time i ask a question. every time i pose a question, i challenge myself to ask a thought provoking question. i challenge myself and i challenge the presenter, meeting group, or whoever the question is directed at to be my level of thought process.

    here is a big hint. asking good questions is one of the better ways to demonstrate that you can critically think about concepts. this is a very good thing to show off. :) especially in interviews, to your supervisors, and to anyone else that will notice.
  • Tuesday, December 11, 2007

    a crazy hackystat idea

    i gotta start heading out of work, but here is a quick crazy idea.

    what if the hackystat user interface lived within something like apache lenya or ning. i don't think that is realistic, but what i kinda like the thought. applying hackystat to social networking instead of applying social networking to hackystat. anyway, that is random thought, but maybe it can jump start some brainstorming about what kinds of specific things software developers want in a "collective intelligence hackystat system.

    Monday, December 10, 2007

    my new favorite company - palantir

    palantir is one of my new favorite companies. not only do they say the right things, i actually like what they do. first, here is what they say.

  • Your Are Brilliant, We are Hiring
  • We stand for elegance, beauty, and ease of use in industries that have traditionally been underserved in all these areas.
  • Analysis is fundamentally collaborative. When looking for signals in gigabytes of noise, isolation is a recipe for missed discoveries. An organization that can share the right information between the right people is far more intelligent than the sum of its parts.
  • we live and breathe good software. Even as we tackle problems from the domains of finance and government, we are driven by our roots in technology. We are top technical minds who get things done.
  • It's also true of our people - one brilliant coworker is worth ten above average ones.
  • Change the world through new ways of exploring information.
  • Brilliant, social teammates who are some of the smartest people you'll meet anywhere.
  • Flat, meritocratic structure means the best idea wins. Make a big difference on your first day.
  • Stock options in a company with real revenue and tremendous growth potential.
  • Beautifully engineered products that our customers love.
  • Well-funded startup environment with full salary and benefits.


  • wow, that is some cool statements. imagine if they actually believe and do those things. anyway, palantir is pretty cool.

    now on to the thing that i beginning to love about them. check out this blog post: Palantir: so what is it you guys do?. there is too much in the blog post to summarize. but here are some interesting points.


  • we specialize in analysis.
  • analysis is everything necessary to extract insight from information
  • the analyst (human) should be in control.
  • summarize large data sets (machine learning, statistics)
  • visualize large data sets
  • iterate rapidly - ask question, get answer, ask a variant question
  • collaborate with other analysts effortlessly
  • hardest part is data modeling, figuring out what data type are relevant to a domain.
  • Data modeling, data summarization, and data visualization are the core disciplines for approaching large data sets. Human-driven queries, rapid iteration, and collaboration are multipliers, taking the power unlocked by the core disciplines to the next level. When these pieces are brought together in a coherent system, the result is in an analysis platform both very generic and very powerful.
  • welcome to the future of analysis.


    WOW! this is the kind of stuff i have been thinking about for hackystat. we need some palantir approach to our hackystat analyses. this realization of sophistication is what makes analyses useful. information (the data) is meaningless if we don't understand it. to me, palantir is all about making meaning. that is a cool concept. they are insight providers.

    so... i think hackystat could use several things:
  • interoperate between levels of information
  • summarize large datasets, but also provide fine grain details when needed
  • make connections to others in the project team.
  • provide feedback
  • provide context
  • data mine to help find useful information
  • iterate over analyses and reports
  • share charts, comments about charts, usefulness context,
  • publish reports based on analyses
  • push out awareness
  • etc, etc.

    i like the idea of a palantir approach to hackystat. hackystat provides information. a palantir approach to the analysis provides meaning. simple charts and tables or even fancy charts and tables just doesn't cut it.

    palantir has an approach, i wonder if it will work. if it does, it will be the future of analysis.
  • re: ICS or EE

    one of my friends posed this question: "i was thinking about transferring to ICS from EE, what do you think"?

    here is my generic answer:

    companies needs management that understands about software development. in my opinion, a lot of companies are missing that. companies that have that sort of leadership probably has a culture that has good software practices. so, i think that you can grow those skills and use it when you lead your company. but, that is the important part. you need to use it to lead your company. that means creating policies and management procedures that rock star programmers feel that they can thrive in your company. that means understanding how developers think and what drives them. in my opinion, i think only people with software development expertise knows how to do that.

    on the other hand, i think, going the engineering rout is equally important. companies need subject matter expertise. and that is equally important. engineering offers so many other subject matter expertise that ICS cannot provide. (just think if they combined curriculum's. that would be awesome). in my opinion, a rock star would "rock" either major. EE guys are awesome too.

    both opportunities will teach you how to conduct research and to critically think about problems. that is the most important aspect of any masters or phD education. it is the quest to write down your ideas and evaluate them objectively and subjectively that is most important. in addition, growing leadership skills, communication skills, and "knowledge skills" are really important. so one needs to focus on that.

    i don't think picking one major or the other will vastly change your ability to make an impact. its more about what you like and what you enjoy learning. one or the other are fantastic opportunities.

    i just personally like the software engineering problem more than i like something like signal processing. my belief is that software is going to revolutionize the world once more of us attack the essence of software (read the mythical man-month by fred brooks if you don't know what i'm talk about).

    anyway, back to you.. i think you have a wonderful opportunity to learn about solid software engineering practices (from Dr Johnson) and actually putting them in place; by sticking with ICS and using your position to make difference in your company and in the industry. that is just my 2 cents.

    either way, i know it will be an awesome experience.

    Saturday, December 8, 2007

    re: My Resume Objective Is Crap

    austen wrote a interesting blog about his objective statement. based on what he wrote, i have some additional comments.

    how about this instead:

    i value work environments that value developers over products, science over marketing, and innovation over the bottom line. in this environment the development team can write software that is revolutionary and will make a difference. As a software engineer, i strive to create software products of the highest quality and i am determined to find a work environment that matches my enthusiasm.

    well, that encompasses things that i strive for. but, i believe you are interested in similar things

    in fact, one needs to figure out a mantra. now that is a cool idea. what is your personal mantra. thinking for 20 seconds, my mantra is "quality innovation" or maybe its something like "world changing" . haha something like that. i think a problem that i am currently struggling with is trying to find my personal mantra.

    once we figure that out.... one needs to determine whether the work environment has people with similar aspirations, mantras, motivations, determinations, skill, etc. in addition, whether those same things are valued by the company by creating policies, organization structure, projects, etc where those people can strive. it is all connected. people and companies need to understand that. companies like fogcreek, google, and it seems like even atlassian is getting there.