Keep Coding

Moving...

2010-03-10T19:39:00.000-08:00

I'm moving from blogger to wordpress and I got a new domain! In the next few days I'll be fixing some issues from the imported data, cleaning up old stuff and theme tuning. Here's is the new url: http://marcovaltas.com

Meanwhile I'll keep the old posts in blogger if eventually someone has a bookmark or a post shows up in Google search.

Best,

Marco Valtas.

Managing Geeks

2009-09-20T15:25:00.000-07:00

I just read this blogpost from Patrick Kua which will lead you to this. As Patrick I don't agree with all, but some points got my attention.

Geeks are trained in logic and ordered thought.

With these skills you can't make BS decisions, if does not make sense they will detect it easily. When your IT resists to some business decision, is better check out what is happening, they can function as compass against illogic paths.

Respect is money

In the IT world respect is money. Note that all open source projects work this way. You can't earn respect with nice talk, smiles or other common social weapons. IT guys will respect those who knows what they are talking about and are humble for those things they don't know. Respect is earned with work, good work and team work. Force will just make them shut and let you, or your project, drawn alone. With respect they will follow you, work late and keep you project the best as they can.

IT can't fake work

This was something I never notice but, IT can't fake work (in general). So, anything that sounds like fake you be detected and can't be done at all. Usually other departments (who asked for the impossible) will throw stones on IT, telling that they are slowing the project. In fact, when I saw this, the root cause was a project without value, something to cover incompetence or IT is full and can't deliver at the velocity asked. Is a good thing check what has been asked to IT. In my experience, IT will prioritize by it self. They will not respect illogic priorities and the will organize as they see the business plan. It's very important to make it clear for IT where your business plan is pointed to, if you do, they will take care of projects that point in the same direction.

Conclusion

Well, these are just observations I made through the my years in IT. IT is a very important staff, usually they can spot problems very fast and a good understanding of the minds of IT is a good tool to lead your business to success.

Blog Quick note - disabled automatic line breaks

2009-09-16T06:21:00.000-07:00

I just disabled the automatic line breaking option in blogger. Probably old posts will be a lot messy but I can't rewrite them all. Blogger is not the best tool for blogging but is what I have.

The reason for the change is that I'm now writing the posts using Markdown and Vim which makes me a lot more productive in writing for this blog and I hope to write a lot more now.

Best. Marco.

Software Philosophy - Refactoring Paradox

2009-09-16T06:09:00.000-07:00

Another day I was talking with a colleague about the Interpreter Pattern to solve a implicit language problem. Martin Fowler describes a pattern called Query Object to solve this, which is a interpreter focused on database queries.

Refactoring as the best technique

Code Refactoring is, after learn a language, is the most important thing a developer should know. Refactoring is what really reveals what soft means in software. When building a bridge is practically a one shot, once the foundations are ready you can´t improve them as the construction continues. But with software, you usually improve the foundations as you go, mostly because nobody knows where is the other side of the bridge you are building.

Refactoring paradox

Refactoring has a implicit paradox, you have to factor before refactor. You can´t, at first, use such good technique without make mistakes a priori. This fact became clear when we had to refactor a code which was growing to cope with combinations of SQL queries. In the book Refactoring to Patterns this problem is described as Replace Implicit Language with Interpreter.

But if you think about it you have to code the queries and many methods to see what implicit language is taking form. Probably you will not foresee the language, if you try to predict such implicit language many anti-patterns and bad practices will rise, such as Premature Optimization, YAGNI and etc. For this particularly problem, if you try to code the solution at first, you will end with a criteria object and without any value from your code since this generic approach already exists.

Frederick Brooks in the Mythical Man-Month wrote about the Pilot System. This idea implies the construction of a throw-away when coding a new system. He argues that if you not plan you will throw-away anyway. This idea agree with the refactoring paradox, you can´t refactor before factor and there´s no need for refactoring a good code, but the chances to get it right on the first try is minimal, I will make a bold statement that is impossible to get it right on the first try.

Conclusions

Brooks is right, plan to a throw-away, you will anyway, at least part of the code will be replaced.
Don´t fear rewriting, is part of the job, put it on your estimates.
Software is soft don´t try to make it hard, is unnatural.

That´s it.

JPA, a little more complex example.

2009-07-28T05:10:00.000-07:00

This week I was writing a tool to extract reports from Mantis Bugtrack using JPA and EJB3. Since I´m not good in JPQL (sonething like a SQL for objects) the search for examples began. The problem was that all examples were too simple, here´s some notes one a more complex query and how was done. Mantis is a bugtrack tool and allows a bug relationship and custom fields, my report will make use of both. Below is a simplified diagram of the database tables involved in this process. The scenario was 2 projects. The first (I will call it A) will open bugs for the second (from now called B) the bugs will be related. Example: A operator opens a bug in project A, after that n other bugs are opened in project B, the relation between bugs is B are child of A. This relation is explicitly set using Mantis relationship features which will be reflected in our bug_relationship table. The last thing on this problem is that a custom field value should be retrieved too for a pivot transformation. Here's another diagram showing the relationship:

To not extend too much this post I will show a simplified version of the mapped entities. Gets, Sets, packages and imports were omitted for the sake of the reader.

@Entity
@Table(name="mantis_project_table")
public class Project implements Serializable {
 @Id
 private int id;

 private String name;

 @OneToMany(mappedBy="project")
 private Collection<bug> bugs;

    // gets and sets omitted.
}

@Entity
@Table(name="mantis_bug_table")
public class Bug implements Serializable {
 @Id
 private int id;

 @Column(name="project_id",insertable=false,updatable=false)
 private int projectId;
 
 @ManyToOne
 private Project project;

      //gets and sets omitted
}

public enum BugRelationType {
 DUPLICATED(0),
 RELATED(1),
 DEPENDANT(2);

 private final int relationship;
 
 BugRelationType(int rel) {
  this.relationship = rel;
 }

 public Integer toInteger() {
  return Integer.valueOf(this.relationship);
 }

}

@Entity
@Table(name="mantis_custom_field_table")
public class CustomField implements Serializable {
 
 @Id
 private int id;

 private String name;

      // gets and sets omitted

}

@Entity
@Table(name="mantis_custom_field_string_table")
public class CustomFieldString implements Serializable {

 @EmbeddedId
 private CustomFieldStringPK pk;

 private String value;
}

@Embeddable
public class CustomFieldStringPK implements Serializable {

 private static final long serialVersionUID = 1L;

 @Column(name="field_id")
 private Integer fieldId;
 
 @Column(name="bug_id")
 private Integer bugId;
}

Alright, with those entities in hand how we will compose a good JPQL without access the database more than one time? It's possible? Is possible to join all these entities? Those were my questions. The answer is affirmative, but to do so is not straight forward (at least for me) like when I'm writing SQL.

Here's is the final query:

SELECT 
    bt, cs.value  
FROM  
    CustomFieldString cs,  
    CustomField cf,  
    Project ps, IN (ps.bugs) bs, 
    Project pt, IN (pt.bugs) bt, 
    BugRelationship rel  
WHERE   
    ps = :projSource  
    AND pt = :projTarget  
    AND rel.sourceBugId = bs.id  
    AND rel.destinationBugId = bt.id  
    AND cs.pk.bugId   = bs.id  
    AND cs.pk.fieldId = cf.id  
    AND cf = :customField  
    AND rel.relationshipType = :relType

Now just fill up the named parameters and you have it, a fairly complex JPQL query.

That's it.

What is keeping me busy

2009-07-14T19:38:00.000-07:00

Just a small update on stuff I'm doing in the past weeks.

Creating Business Diagrams (BPMN) and Simulations in TIBCO Business Studio
Getting serious in SQL with analytical functions of Oracle 10g
Studying how to write mathematical proofs (book: How to Prove It)
Reading (at last!) the classic Frederick Brooks: The Mythical Man-Month
Reading some of Edsger Dijkstra writings

As you can see my current time in learning is devoted to mathematics, oracle and software development management. In short the mathematical is just improve my skills with logic and reasoning, the oracle study is for my current path to OCP and the management studies is for professional growth.

On the shelve (not at particularly order):

That's it.

Technology Loop Paralysis - Anti-Pattern

2009-05-25T16:29:00.000-07:00

Last Sunday I was in the "Falando em Java 2009" talking with a friend when we got in the subject of new technologies. During the conversation I realized one common anti-pattern that I didn't know about, the Technology Loop Paralysis, TLP for now on.

The TLP has the following major characteristics:

It's seems right, without attention can't be caught.
Once started, break the loop is hard.
Scrambles all current problems, so disguises itself.

So, what is TLP? TLP is just the change of a technology, maybe a framework, a language or something similar to avoid difficulties in development. The catch is, when a new technology is added all kinds of problems will arise.

TLP Examples:

Sometime ago me and another developer had problems in delivering changes to a legacy Java system, the system was poorly designed and any change could break a lot of unrelated features. When the difficulties were explained the manager asked "If we use PHP? Can we fix?". This is a classic TLP. The problem wasn't with the language, any effort had to be directed in refactoring not in language change. Experienced developers will notice that this is trap easy to fall, if the developers were inexperienced and more comfortable with PHP they could fall on this trap.

Another TLP example. In a bugtrack system, developers didn't fill the forms right. Status, ETA and other fields were empty, they also didn't close issues properly. Relations were poorly added and resolutions were not written accordingly. Then, some reports were created to control issues, SLA and so on, but the data was not good. The solution proposed? Change the bugtrack system, but the system wasn't the problem, change it would only disguise all current problems.

TLP Effect:

New technologies will not solve any team, managing structure, unrealistic expectations and similar problems. Once TLP started these problems will disguise as implementation problems.

TLP Name Explanation:

Technology: Any support system, framework, language or build block to a system.
Loop: The fact that when faced with problems, the team or management choose change the technology without know root cause of the problems faced.
Paralysis: The problems can't be solved and despite the feeling that something is been solved it's not.

TLP Forces:

Technologies change and grow more rapidly than our systems, so before a team can finish one project is common that the supporting technologies have new and better versions.

Inexperience and anxiety for new features can lure the team and management to adopt every brand new technology around.

TLP Solution:

Experience to avoid fall in TLP in the first place. During a TLP, the team can stop looping and stick with a technology until it finishes the project. After finished, the upgrade should be considered harmful unless all implications are well understood.

Why Developers shouldn't work long hours.

2009-04-25T16:46:00.000-07:00

In fact this post should be named "Why anyone shouldn't work long hours". Face a problem for long hours, fighting a deadline it will not help, this should common sense, if not, try read about "Sleep-dependent memory consolidation".

My point here is not about problem solving, performance and accidents, I think these topics are well documented. But I want to add just one little information on why "long hours" are a problem.

What happened to me this week is that I had to solve some personal matters, in fact, renew my drivers license. It took me almost a half of day and got me thinking. How come I can do this if my project was late? Stay with a expired driver license? Work until midnight? In either case I will get unsatisfied and worried, with this kind of mood how a good work can done?

Some arguments against a "long hour" work is that people have to live outside the cubicle, but in fact there's some things more practical. We can't cope with all things we need to do if we are in a "long hour work" policy, there's no time.

Developers have to work on a normal scheduling, if not, the consequences are terrible. A tired developer with fall in the easy solution to reach the deadline. This will lead to a less extensible code, that will lead to problems in module integration, that will make the final period of development a bug hunt and more long hours to find those. Well, what can happen if a you add tired developers, buggy code and deadline approach? Something has to give up. You can't make the developers more efficient with more long hours, the bugs will stay in the code until someone removes them, so the most common to give up is the deadline. The "long hour" solution turns to be longer that you expect.

That's it.

Inside Steve's Brain

2009-02-23T11:56:00.000-08:00

I just finished the book Inside Steve's Brain, good book specially if you like Apple products like me. One tip about this book is not read the final notes of each chapter, mostly they are shallow summaries out of context. The book detail some decisions and modus operandi of Steve Jobs that I personally found very interesting, first because in most of times I agree (in my humble opinion), and secondly because it works, for everyone who knows about the computer market can see that some actions made by him are "default" on computer industry nowadays. Here some points made in the book:

Small and good teams
Every one involved from begining to end
Avoid the "more features" trap, keep it simple
Prototype a lot
Keep focused, your user is the focus, but he/she doesn't know what really want

Small and good teams Probably is no news the that any team has to be assembled with good professionals, and adding more people don't help. And big teams will suffer with the n(n-1)/2 rule for communication channels, if your team is too big probably there's isn't a single view of what should be done. Everyone involved from begin to end Well, the schema software engineers make the software, the designers the design, the marketing people the marketing and so on, doesn't work. In fact, I really believe that didn't work forever. Just ask yourself, in any point of your profession if people told you the big picture, not all the business plan, but a little more, things would be done differently? Maybe a focus on modularity here and there, or a external configuration system or similar stuff. Probably, in my case, things will need less duct tape if a more broad view is attached with the requirements. Avoid the more features trap, keep it simple Too much people, including myself wrote about the KISS principle. Prototype a lot I believe this is a true way to get good products. No body will get it right on the first shot specially on a complex system. Software Prototyping is a valid approach in development, but it seems that in Apple every part of a product is prototyped. In case you're not convinced, they even prototyped the Apple Store. Keep focused, your user is the focus, but he/she doesn't know what really want Nobody will disagree that focus is important, but some will disagree that consumers don't know what they want. I'm part of a group that believes consumers (users in my case) feel what they want but are incapable of expressing it correctly, why? Because they don't know the tools you have, the possibilities you have in hand to build something. Is more safe to think they can feel a good experience and tell you "I'm really comfortable using this software", than "I need a checkbox here, a list there and etc." People are subject to the Paradox of Choice and it seems that Steve Jobs knows this very well. Conclusion This is just a glimpse of what I learned from the book, somethings reinforced my convictions others made me think. It's a very light reading and a good book for who works in this area or not.

Running Selenium with TestNG

2009-02-18T16:00:00.000-08:00

This week I had to rewrite a bunch of functional tests, mostly Selenium stuff. When I think about Java Unit tests the first thing that comes to my mind is the JUnit framework. All tests that I have written until past week were with JUnit, and was using it to run Selenium. But I got one more requirement, since I was rewriting all tests I would like to change environments (development and pre-production) without dealing in the code where was running the tests. The problem was a simple one, just write configurations with URLs, expected values and so on, for each of my environments. A trivial example is using different URLs in selenium for the same tests. Well, how do this with JUnit? First approach, using JUnit 3 In JUnit 3 (old one) you have Test Suites. Programatically you have a chance to read a properties file and give the information to your test classes. Well, this could work if I had not the principle: "Do not write test code if isn't a test." When I caught myself creating a "little" framework to test my application I realized "I'm in the wrong way". Not that you should never code a "little" framework to simplify even more the tests, but i hadn't the time to do it. Second approach, using JUnit 4 Well, JUnit 4 is more flexible, newer and uses annotations; and I didn't explored it well. After some digging i found JUnit's Parameterized Tests, but again I had (bear with me in this one) "to code, code to test code". And one more thing, JUnit 4 have Test Suites but, AFAIK, you only have a annotation version to define the classes for a suite. JUnit wasn't solving the problem After some thought i had two choices, forget about environments, or spend some time coding new stuff that wasn't tests or my application. Justice has to be made, JUnit is a excellent test framework but I think my objectives weren't too "Unitwise". TestNG, next generation? I heard about TestNG about a year ago and didn't gave it enough attention, I was happy with JUnit. But, everyday is a new day and I had a problem, so I started to read about TestNG. I was skeptic at first, thinking "maybe this can do it, but I don't have time to learn it all". I was wrong, happily wrong. TestNG, next generation! I have to confess, there's nothing to write here, maybe just some code. Remember the problem? "Two environments, one test, test it all". First, let me show the test example.

package com.keepcoding.test;

import static org.testng.AssertJUnit.*;
import org.testng.annotations.*;
import com.thoughtworks.selenium.*;

public class Google {

 private Selenium selenium;
 
 @BeforeClass
 @Parameters({"selenium.host","selenium.port","selenium.browser","selenium.url"})
 public void startSelenium(String host, String port, String browser, String url) {
  
  this.selenium = new DefaultSelenium(host, Integer.parseInt(port), browser, url);
        this.selenium.start();
        this.selenium.open(url);
 }
 
 @AfterClass(alwaysRun=true)
 public void stopSelenium() {
  this.selenium.stop();
 }
 
 @Test
 @Parameters({"search","expected"})
 public void googling(String search, String expected) {
  try {
   selenium.type("name=q", search);
   selenium.click("name=btnG");
   selenium.waitForPageToLoad("3000");
   assertTrue(selenium.isTextPresent(expected));
   
  } catch (SeleniumException e) {
   fail(e.getMessage());
  }
 }
}

This is a simple test, will start Selenium, open a URL (to be defined), type something (to be defined) and finally check if a text is present. Some notes:

@AfterClass is defined with "always=true" because Selenium has to stop, even on test a failure.
Selenium commands are surrounded with a try/catch because I want to avoid ERROR statuses, better a FAILED status if Selenium fails.

Now let's look on TestNG configuration file:

Here I defined all parameters and what tests should run, as you probably guessed the test search on Google for "Keep Coding" and checks if this blog URL is in the results. The key thing is not the test itself, but it's setup. We have defined Selenium to open a certain URL and if we want change the URL is a simple case of editing the XML file. Using Ant to test all environments Finally, how test all environments? First, create another TestNG configuration file, change the parameters accordingly, and use Ant to run it all. Here's a build file example:

This is a simple ant file, the highlighted part shows where you can have as much environments you want, for each one create a file changing parameters, skipping (not in production) tests and so on. TestNG will run one file after another for you. Conclusion TestNG is really impressive, from very simple tests to complex environment tests, you have in hand a set of options to make it happen. When you mix TestNG with Selenium and Ant is possible to build very complex testing. TestNG goes beyond the "units" and provide what you need to think about "collections".

Web Services Versioning

2009-02-08T10:56:00.000-08:00

I had a problem last week and it's about Web Services versioning. I read several posts about this issue but I didn't found a simple solution. Most solutions are somewhat complex using UDDI or proprietary based, but my problem was very simple stated like "Once published a Web Service you cannot change it in a way you will brake your clients". So, with this constraint how to evolve your application without break your API or getting too duplicated code? Use URLs with wisdom First develop a strategy for your services URLs, put the version of your service inside the URL. In my case:

http://myapphost/services/v[MAJOR]_[MINOR]/[SERVICE]

Here's a example:

http://myapphost/services/v1_0/SomeGoodService
http://myapphost/services/v1_1/SomeGoodService

Note that you should keep the name of the service, this is very important! Same service same name. You don't want SomeGoodService, SomeGoodImprovedService, SomeGoodImprovedFastService to be the names. For the WSDLs files I use the same reasoning.

http://myapphost/wsdl/v1_0/SomeGoodService.wsdl

Or you can use dynamic WSDLs using functionality of the webservice framework, I use JBoss so I can provide a WSDL dynamically.

http://myapphost/services/v1_0/SomeGoodService?wsdl

Using a URL strategy is a good way to tell your client what version of your web service is in use, it's a fixed endpoint and remember that this URL will exists for a very long time. Expose simple versions of your objects When you have complex structures or dependencies on your objects this will be reflected inside your client code, so provide a simplification to your client. Such simplification pays off, avoiding complexities of XML type mapping is good because you don't know in what language your client will be developed. Making deserialization easy is a good thing. For a example in my current app I had a object using some joda time types, if I exposed such object to a web service lot of unnecessary complexity would flow into the client. To fix this I created a simple version of it, here's a simple method signature simplification: void setBeginHour(LocalTime beginHour); to void setBeginHour(Calendar beginHour); Some dirty details Well, enough with recommendations (language and application server agnostic), now let's look into some real implementation. My environment is Java 5 with a JBoss 4.2.2 as application server. I will use a simple web service called AgeService, given a birth date it tells how years old is the client. Here's the package layout of my web service:

br.com.svvs.model - Domain objects for internal use.
br.com.svvs.service - Web Service implementation package.
br.com.svvs.service.model - Simplified versions of domain objects.
br.com.svvs.service.util - WSTypeAdapter should translate domain to ws objects and vice-versa.

Now let's look into the Birth class:

package br.com.svvs.model;

import org.joda.time.DateTime;

public class Birth {

 private DateTime birthDate;

 public DateTime getBirthDate() {
   return this.birthDate;
 }

 public void setBirthDate(DateTime birthDate) {
   this.birthDate = birthDate;
 }

}

It's a very simple class, the problem is it uses DateTime from joda-time and I do not want expose such implementation detail. To hide this detail let's create a simplified version just for use on my service. Here's is the WSBirthV1_0:

package br.com.svvs.service.model;
import java.util.Calendar;

import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;


@XmlRootElement(
   name="wsBirth",
   namespace="http://wsbirth_1_0.model.service.svvs.com.br"
)
@XmlType(name="wsBirth")
public class WSBirthV1_0 implements WSBirth {

 private Calendar birthDate;

 public void setBirthDate(Calendar birthDate) {
   this.birthDate = birthDate;
 }

 public Calendar getBirthDate() {
   return birthDate;
 }

}

WSBirthV1_0 changes the signature to a simple Calendar object which most XML mappers know how to convert. Note WSBirthV1_0 implements WSBirth, this a marker interface we will use it in WSTypeAdapter, let's look at the code:

package br.com.svvs.service.util;

import org.joda.time.DateTime;

import br.com.svvs.model.Birth;
import br.com.svvs.service.model.WSBirth;
import br.com.svvs.service.model.WSBirthV1_0;

public class WSTypeAdapter {

 public static <T extends WSBirth> T convertTo(Class<T> type, Birth wsbirth) {
 
   String typeClassName = type.getSimpleName();
 
   if(typeClassName.equals("WSBirthV1_0")) {
     return type.cast(convertToWSBirthV1_0(wsbirth));
   } else {
     throw new RuntimeException("Don´t know how to convert " + typeClassName);
   }      
 }

 private static WSBirthV1_0 convertToWSBirthV1_0(Birth birthDate) {
   WSBirthV1_0 wsbirth = new WSBirthV1_0();
   wsbirth.setBirthDate(birthDate.getBirthDate().toCalendar(null));
   return wsbirth;
 }

 public static <T extends WSBirth> Birth createFrom(T wsbirth) {
 
   String versionName = wsbirth.getClass().getSimpleName();
 
   if(versionName.equals("WSBirthV1_0")) {
     return createFromWSBirthV1_0((WSBirthV1_0) wsbirth);
   }
   else {
     throw new RuntimeException("Don´t know how to produce Birth from " + versionName);
   }  
 }

 private static Birth createFromWSBirthV1_0(WSBirthV1_0 wsbirth) {
   Birth birth = new Birth();
   birth.setBirthDate(new DateTime(wsbirth.getBirthDate().getTimeInMillis()));
   return birth;
 }


}

In this class, using generics, we can provide a simple API to convert to and from ws objects. Note the use of our marker interface WSBirth. It's a utility class with static methods, we can grow our private methods for more and more versions. And at last, here's the service:

package br.com.svvs.service;

import javax.ejb.Stateless;
import javax.jws.WebService;

import org.jboss.wsf.spi.annotation.WebContext;
import org.joda.time.DateTime;
import org.joda.time.Interval;

import br.com.svvs.model.Birth;
import br.com.svvs.service.model.WSBirthV1_0;
import br.com.svvs.service.util.WSTypeAdapter;

@Stateless
@WebService(
   name="AgeService",
   serviceName="AgeService",
   targetNamespace="http://age_1_0.service.svvs.com.br"
)
@WebContext(
   contextRoot="/services",
   urlPattern="/v1_0/AgeService"
)
public class AgeService_1_0 {

 public String tellAge(WSBirthV1_0 wsbirth) {
 
   Birth birth = WSTypeAdapter.createFrom(wsbirth);
 
   System.out.println(birth.getBirthDate());
 
   Interval i = new Interval(birth.getBirthDate(), new DateTime(System.currentTimeMillis()));
 
   return "You have " + i.toPeriod().getYears() + " years old.";
 }

}

Discussion At first it seems to much work but I disagree. When you have to maintain old web services and did not planned for code evolution this could raise much more work, usually with a copy and paste solution (happened to me). Some good points about this solution:

You can evolve your code without fear of breaking the old service, just follow the conventions.
Isolated conversion from domain to ws objects, you don't forward ws objects inside the domain.
Control of what is exposed and how, simplified client development.
It works.

Some bad points:

Messy names like ServiceV1_0, ServiceV1_2, WSObjectV1_0 ...
Care should be taken on annotations and namespaces.
If you have complex domain objects, you have to create a simplified version of each object used in your service.
Not simple enough, I still think that could be simpler.

Some thoughts about the real thing.

2008-12-22T17:23:00.000-08:00

It's been some time that I'm feeling disappointed with software development, probably any developer felt this some day in his (her) career. Software is hard, hard to get it right, and things get pretty ugly when business requirements take place. Despite all work done in better software management and development, still, get it right is hard. We surely have people who can make good software, I will cite ThoughtWorks as a example. But it seems to me that the distance from the day by day software from what is posted in PlanetTW is enormous, at least by my experience. One fact is that every company has/need some in house software, the big ones and the small ones. They have IT managers which were elected to hire developers in the field to write some code. The managers don't know what correctness means for software, and the developers know that with enough tries something eventually will work. Both far from know who was Edsger Dijkstra and understand the reasons why he wrote:

Unfathomed misunderstanding is further revealed by the term "software maintenance", as a result of which many people continue to believe that programs - and even programming languages themselves - are subject to wear and tear. EWD1036.

This is not just about shooting stones on everybody roof since my is made from glass too, I consider myself one more developer trying to make better code everyday. I'm probably not alone when I mention that the "best things I wrote were in spare time", and that was not because it was off-work code, it was work code, real work code, but I knew what needed to be done and dedicated myself to discover "how" should be done. Time not allowed in many companies. The real world of today's developer seems a schizophrenic one. In one hand writing, modeling, saying, in the other hand not knowing if wrote, modeled or spoken the true. Still amazes me how much "It works!" I hear in software development. Best.

Ant Power, Cruise under Control and Selenium Fuel (part 4): Managing dependencies in Cruise Control

2008-11-20T06:41:00.000-08:00

On my last post I mentioned about how to configure Cruise Control to build projects that depends on another project's artifacts. But I described only the configuration and how to find the latest artifact built. Now is time to show the build files of the projects and how they know what artifact to include. This can be simple as add one more classpath to Ant. But I have one more issue to solve, branches. The problem here is, as many of you already know branches are used to split the development tree so you can have all sort of things developed in parallel, like new features, bug fixes and so on. In my case we usually have a main development tree and a bug fix tree. Of course I wanted that both trees were subject to Cruise Control, so the solution was call the bug fix project "project-release" and the main project just "project". When you have such structure you have to merge both trees from time to time, the problem was if my Ant build file in the release points to a release artifact and the trunk points to a trunk artifact, when they get to merge we always will have to take care of the difference. Not a good thing to do. This was extremely easy to solve because Cruise Control has a build call layout very good. Let's see how Cruise Control starts a build. First it sits and wait for a trigger, usually a commit on the SVN tree. After that if decides (it can subject to Queuing and Veto) to build the project, it calls a build file which has the name build-project.xml and inside that file another Ant call is made to actually run Ant on the project's build file. The key thing is, you should put the differences on a trunk build or branch build inside this first build file. There, in my case, I define which artifact will be included in the path. Let's see for one project the Cruise Control configuration file and both build files. Here for the trunk project:

    <schedule interval="60">
    <ant anthome="/usr/local/cruisecontrol/apache-ant-1.7.0"
    buildfile="build-project.xml"
    target="build"
    uselogger="true"
    usedebug="false"/>
  </schedule>

And now for the branch project:

    <schedule interval="60">
    <ant anthome="/usr/local/cruisecontrol/apache-ant-1.7.0"
    buildfile="build-project-release.xml"
    target="build"
    uselogger="true"
    usedebug="false"/>
  </schedule>

The only difference if that Cruise Control calls build on different files for the branch project and for the trunk project. In these files I configure properties suited for each project. Here is the example of both files:

<!-- build-project.xml -->
<!-- for the project (trunk) -->
<target name="build">
<svn>
  <checkout url="http://localhost/svn/project/trunk" revision="HEAD" destPath="." />
</svn>
<ant antfile="build.xml" target="build-all">
  <property name="moduleA.dir"    value="/var/cc-sandbox/artifacts/moduleA/latest" />
  <property name="moduleB.dir"   value="/var/cc-sandbox/artifacts/moduleB/latest" />
</ant>
</target>

<!-- build-project-release.xml -->
<!-- for the project-release (branch)-->
<target name="build">
<svn>
  <checkout url="http://localhost/svn/project/branches/release_1_0" revision="HEAD" destPath="." />
</svn>
<ant antfile="build.xml" target="build-all">
  <property name="moduleA.dir"    value="/var/cc-sandbox/artifacts/moduleA-release/latest" />
  <property name="moduleB.dir"   value="/var/cc-sandbox/artifacts/moduleB-release/latest" />
</ant>
</target>

Now, since Ant inherits this properties I have to just create one single build file which is the same for branch and trunk. For example:

  <path id="internal-dep-classpath">
   <!-- depends on moduleA project -->
   <fileset dir="${moduleA.dir}" >
     <include name="*.jar" />
   </fileset>
   <!-- depends on module B libraries -->
   <fileset dir="${moduleB.dir}" >
     <include name="*.jar" />
   </fileset>
 </path>

That's it.

Ant Power, Cruise under Control and Selenium Fuel (part 3): Projects that depends on others artifacts.

2008-11-18T16:27:00.000-08:00

One thing I had to do when configuring CruiseControl for the first time was manage the dependencies between projects. It happens that my application project was a EAR of three other projects (see the diagram below). Each of the modules (A,B and C) has a development tree and the JARs and WARs are assembled into the final EAR. For this reason I needed a way to identify the latest artifact of each project so it can be used to assemble the EAR. Another thing that I had to manage is control the dependencies in the build loop, as you can see Module C depends on Module B, and Module B depends on Module A. If someone changes the Module A all other modules should be recompiled and assembled to ensure that the build is still possible. Now let's see one solution for this scenario.

Projects that depends on others artifacts

First let's look into Modules inter dependencies. CruiseControl has a option called veto and is explained in this wiki entry. I took some time to understand how it work but it worked well. The key to understand the veto parameter is that instead of the "Module C depends on Module B" you have to think "Module B inhibits the build of Module C and Application EAR", in my case "Module A inhibits the build of Module B, Module C and Application EAR". Another thing to note is the buildstatus parameter, which tell CruiseControl what triggers a build. In my case "Application EAR should be built if something changes on Module A, Module B, Module C or Application EAR". Below is listed the modificationset of the projects.

<!-- module A -->   
<modificationset quietperiod="10">
  <veto>
    <triggers>
      <svn localworkingcopy="projects/moduleB" />
    </triggers>
    <buildstatus logdir="logs/moduleB"/>
  </veto>
  <veto>
    <triggers>
      <svn localworkingcopy="projects/moduleC" />
    </triggers>
    <buildstatus logdir="logs/moduleC"/>
  </veto>
  <veto>
    <triggers>
      <svn localworkingcopy="projects/applicationEAR" />
    </triggers>
    <buildstatus logdir="logs/applicationEAR"/>
  </veto>
  <svn localworkingcopy="projects/moduleA"/>
</modificationset>

<!-- module B -->
<modificationset quietperiod="10">
  <veto>
    <triggers>
      <svn localworkingcopy="projects/moduleC" />
    </triggers>
    <buildstatus logdir="logs/moduleC"/>
  </veto>
  <veto>
    <triggers>
      <svn localworkingcopy="projects/applicationEAR" />
    </triggers>
    <buildstatus logdir="logs/applicationEAR" />
  </veto>

  <buildstatus logdir="logs/moduleA" />

  <svn localworkingcopy="projects/moduleB"/>
</modificationset>

<!-- module C -->
<modificationset quietperiod="10">
  <veto>
    <triggers>
      <svn localworkingcopy="projects/applicationEAR" />
    </triggers>
    <buildstatus logdir="logs/applicationEAR" />
  </veto>

  <buildstatus logdir="logs/moduleA" />
  <buildstatus logdir="logs/moduleB" />

  <svn localworkingcopy="projects/moduleC"/>
</modificationset>

<!-- application EAR -->
<modificationset quietperiod="10">

  <buildstatus logdir="logs/moduleA" />
  <buildstatus logdir="logs/moduleB" />
  <buildstatus logdir="logs/moduleC" />

  <svn localworkingcopy="projects/applicationEAR"/>
</modificationset>

With this configuration we have perfect build order. Now, as I mentioned, one project depends on it's predecessor, so Module B requires the artifact of Module A. To achieve this is just add in the build.xml file the path of the last artifact of Module A. But what is this path? When CruiseControl creates an artifact will use a timestamp directory and to add this directory to the build environment in Ant was (for me) complex. The solution was use the good and old symbolic link on Unix file systems. This pattern is not new, just add a symbolic link to current directory of interest, in this case the last artifact directory. So I've written another simple Perl script that does the job.

#!/usr/bin/perl
# vim: expandtab sw=2 ts=2 bg=dark
# this script links the latest artifact
# to a latest directory so we can use
# the artifact as dependence of another
# project.
use strict;
use warnings;

my $ARTIFACT_DIR = "/var/cc-sandbox/artifacts";
my $PROJECT_NAME = $ARGV[0];

die("Missing project name.") unless $PROJECT_NAME;

my $W_DIR = $ARTIFACT_DIR."/$PROJECT_NAME";

opendir(DIR,$W_DIR) or die $!;
  my @dirs = sort(grep { /^\d{14}$/ } readdir(DIR)); # list all dirs, sorted.
closedir(DIR);

# exit if we don't have artifacts.
exit if $#dirs == -1;

# latest directory is on the bottom of
# this array.
my $latest_dir = pop @dirs;

# if we have an latest link lets remove it.
if(-l "$W_DIR/latest") {
  unlink("$W_DIR/latest");
}

# now we create de link to the latest artifact.
symlink("$W_DIR/$latest_dir","$W_DIR/latest") or die ($!);

exit;

This script receive as parameter the project name, enters the artifacts directory and creates a symbolic link to the lastest directory called "latest". To run this script just put it in the section publishers of CruiseControl config.xml. Here's a example for Module A project.

<publishers>
  <onsuccess>
    <artifactspublisher dest="artifacts/${project.name}" file="projects/${project.name}/target/moduleA.jar" />
    <execute command="bin/link_latest.pl ${project.name}" />
  </onsuccess>
</publishers>

After CruiseControl artifactspublishers task run I use execute task to create the link, now I will find the last artifact easily and any build.xml can refer to this directory. That's it.

Ant Power, Cruise under Control and Selenium Fuel (part 2): Removing old artifacts from CruiseControl.

2008-11-18T06:03:00.000-08:00

One thing I had to do when using CruiseControl is remove old artifacts. Since my CruiseControl starts a build on every commit in times of fast development this causes the creation of lots of artifacts. You probably don´t want to keep all of them since your server disk will get full. Looking at CruiseControl configuration I found a parameter for deletion of artifacts, but it is limited to check the age of an artifact and people said that this configuration assumes your artifacts inside the log directory. When I had CruiseControl setup my artifacts directory is outside of logs/ (I used the config.xml example that comes with CruiseControl and there the artifactpublisher is configured to a outside directory) so use deleteartifacts wasn´t a option. Exist a wiki entry about this but also deletes artifacts based only on age. This is not what I want. If a project doesn´t build for a long time all artifacts will be removed and no last version of the artifact will be available. What I did is just code a simple Perl script that does the job based on age but keeps some artifacts.

Removing old artifacts from CruiseControl

This script should be run by cron at any interval rate you want. There three parameters configurable, the first (MAX_AGE_IN_DAYS) is very clear, how many days old a artifact should be to be removed. The second (MIN_ARTIFACTS_TO_KEEP) tell the script to keep some artifacts even if they´re old enough to be deleted. And the third (ARTIFACTS_DIR) points to your artifacts directory. The script is pretty simple but solves my problem, keep some of the artifacts while removing too old ones. Here´s the code:

#!/usr/bin/perl
# vim: sw=2 ts=2 expandtab bg=dark
# this script removes old artifacts from cruisecontrol
# to avoid full disk. You can configure a min number
# of artifacts to keep and how many days old a artifact
# is considered too old and should be removed.
# Marco Valtas - 2008.
use strict;
use warnings;
use File::stat;
use File::Path;

################
# configuration session.

# artifacts older than this will be removed.
use constant MAX_AGE_IN_DAYS  => 10;
# keep at least this number of artifacts.
use constant MIN_ARTIFACTS_TO_KEEP => 5;
# where we find the artifacts?
use constant ARTIFACTS_DIR => "/var/cc-sandbox/artifacts";

# end of configuration
################


# open artifacts dir to list all projects.
opendir(ART_DIR,ARTIFACTS_DIR) or die $!;
 # list all projects, skip "." files.
 my @projects = grep { /^[^.]/ } readdir(ART_DIR);
closedir(ART_DIR);

# exit if no project was found.
exit if $#projects == -1;

# foreach project, we dig to find
# all artifacts directories.
foreach my $project (@projects) {

 # project artifacts directory.
 my $W_DIR = ARTIFACTS_DIR."/$project";

 opendir(PROJECT_DIR,$W_DIR) or die $!;
   # list all dirs, sorted so old dirs are first.
   my @artifacts = sort(grep { /^\d{14}$/ } readdir(PROJECT_DIR));
 closedir(PROJECT_DIR);

 # skip this project if not enough artifacts were found.
 next if $#artifacts < MIN_ARTIFACTS_TO_KEEP;

 # for each artifact found (skipping the ones we want to keep)
 # check the age and remove it if necessary.
 foreach my $idx (0..$#artifacts - MIN_ARTIFACTS_TO_KEEP) {
 
   # how many days old is this directory?
   my $stat = stat($W_DIR."/$artifacts[$idx]");
   my $days_old =  int((time - $stat->mtime) /86400);

   # remove the artifact dir if is too old.
   if($days_old > MAX_AGE_IN_DAYS) {
     rmtree($W_DIR."/$artifacts[$idx]");
   }
 }
}

That´s it.

Ant Power, Cruise under Control and Selenium Fuel (part 1): Can we run selenium?

2008-11-17T16:54:00.000-08:00

In the past weeks I'm configuring a Continuous Integration environment with CruiseControl. It wasn't a easy task. CruiseControl lacks from updated documentation and you have to know Ant very well (in my view) to get along with it. Since Ant isn't a easy thing to learn (again, in my view) I had some hard time putting all to work. In this series Ant Power, Cruise under Control and Selenium fuel, I'll try to show some of what I have learned.

Can we run selenium?

One not always in cookbooks is conditional behavior of Ant. Ant is capable of a myriad of conditionals but if you look at the official manual you will not find a good example on how to use it. So let's see how Ant was used in my project. As you probably know, Selenium is a test framework that can emulates a user using a browser. It's a good way to automate acceptance tests. But to run selenium the project has to successfully be deployed on server, so how to know if the project was deployed?

First: Learn target depends

Ant can establish dependencies between targets, here a simple build.xml file.

<project name="target_depends" default="build">
    <target name="setup">
            <echo message="initial setup." />
    </target>
    <target name="build" depends="setup">
            <echo message="building..." />
    </target>
</project>

When you run this build file you get this:

$ ant -f a.xml build
Buildfile: a.xml

setup:
     [echo] initial setup.

build:
     [echo] building...

BUILD SUCCESSFUL
Total time: 0 seconds

What happens is that the target build declares a dependency which is the target setup, so Ant takes care to execute setup before build. Nothing too complex until now.

Second: Learn target if/unless

Something clever about Ant is the if/unless attributes on target element, you can abort the execution of a target if you need to. In the next build file we have three targets, the first inhibits the execution of the second.

<project name="target_depends" default="build">
    <target name="setup">
            <echo message="initial setup."/>
            <property name="setup_is_ok" value=""/>
    </target>
    <target name="notify_failed_setup" unless="setup_is_ok">
            <echo message="setup failed."/>
    </target>
    <target name="build" depends="setup,notify_failed_setup">
            <echo message="building."/>
    </target>
</project>

When you run this build file:

$ ant -f a.xml build
Buildfile: a.xml

setup:
     [echo] initial setup.

notify_failed_setup:

build:
     [echo] building.

BUILD SUCCESSFUL
Total time: 0 seconds

You can note that when we set the property setup_is_ok and since the next target on the chain, notify_failed_setup, has a unless attribute this target will not be fired. The catch here is on the depends of the build target, it tell us the order of our build. Ant doesn't have a if/then/else per se, but with depends and if/unless attributes on the target element you can do similar behavior.


<target name="build" depends="setup,notify_failed_setup">

Last: A simple waitfor

For the last step I will use waitfor to know if a project have been successfully deployed, if it is, then we run tests, if is not, I will email me to know that my project isn't deployed.

<project name="cond_demo" default="run_tests" >

  <target name="check_if_page_exists">
    <waitfor maxwait="5" maxwaitunit="second" checkevery="1000" timeoutproperty="page_not_loaded">
      <http url="http://localhost/"/>
    </waitfor>
  </target>

  <target name="email_failed_deployment" if="page_not_loaded">
    <mail
      from="Cruise Control"
      tolist="me"
      subject="Failed to deploy project: ${ant.project.name}"/>
  </target>
                 
  <target name="run_tests" depends="check_if_page_exists,email_failed_deployment" unless="page_not_loaded">
    <echo message="Will run some tests!" />
  </target>

</project>$

Now I have some smart behavior, if Ant can't load the page in waitfor in 5 seconds, it will set the property page_not_loaded, this will validate the call to target email_failed_deployment and invalidate the call to target run_tests. The waitfor task returns ok if the server returns a code 200, if the server returns 404 the waitfor continues until maxwait. Since during deployment (in JBoss) the server returns 404, for my setup is a way to test a successful deployment. The run of this last build file I leave to the reader. One last word, use ant -debug in these build examples to see exactly how Ant works these files. That's it.

Stateful Session Beans, Dependence Injection and @Remove annotation.

2008-10-25T19:49:00.001-07:00

This week I had a doubt about removing Stateful Session Beans (SFSB). Working on a heavy loaded EJB3 application with about 3K objects been created in a single operation I had to watch very closely the performance of my application. The doubt was: "In a Stateful Session Bean which uses another Stateful Session Bean that was injected by the container has to be removed explicitly or the container removes it as soon the first bean is removed?" Sounds complicated but isn't I will give a example. If you have studied the life cycle of a Stateful Bean you already knows that as soon the container creates a instance of the Bean it starts to fulfill the Beans dependencies. Here's a simple Bean called SFSBFirst, here's the interface:

package br.com.svvs;
import javax.ejb.Remote;

@Remote
public interface SFSBFirst {
 void call(String message);
 void remove();
}

And here is the implementation:

package br.com.svvs;

import javax.annotation.PreDestroy;
import javax.ejb.EJB;
import javax.ejb.Remove;
import javax.ejb.Stateful;

@Stateful
public class SFSBFirstBean implements SFSBFirst {

 @EJB private SFSBSecond ssbSecond;

 private String message;

 @Override
 public void call(String message) {
     this.message = message;
     System.out.println(SFSBFirst.class.getName() + ": called(" + message + ")");
     ssbSecond.call(message);
 }

 @Override
 @Remove
 public void remove() {
     System.out.println(SFSBFirst.class.getName() + ": removing(" + this.message + ")");
     //ssbSecond.remove();
 }

 @PreDestroy
 public void preDestroy() {
     System.out.println(SFSBFirst.class.getName() + ": destroying(" + this.message + ")");
 }

}

This bean declares two exposed methods call(String message) and remove(), and ask for the container to inject the bean SFSBSecond. It also annotate a method with @PreDestroy which makes the container call it before garbage collect the bean. It's a simple bean just to check if I have to call remove() on SFSBSecond or not. So let's look on SFSBSecond interface:

package br.com.svvs;

import javax.ejb.Remote;

@Remote
public interface SFSBSecond {
 void call(String message);
 void remove();
}

And the implementation:

package br.com.svvs;

import javax.annotation.PreDestroy;
import javax.ejb.Remove;
import javax.ejb.Stateful;

@Stateful
public class SSBSecondBean implements SFSBSecond {

 private String message;

 @Override
 public void call(String message) {
     this.message = message;
     System.out.println(SFSBSecond.class.getName() + ": called(" + this.message + ")");
 }

 @Override
 @Remove
 public void remove() {
     System.out.println(SFSBSecond.class.getName() + ": removing(" + this.message + ")");
 }

 @PreDestroy
 public void preDestroy() {
     System.out.println(SFSBSecond.class.getName() + ": destroying(" + this.message + ")");
 }

}

Still a simple bean too, and now let's look at the test code:

import javax.naming.InitialContext;
import javax.naming.NamingException;

import br.com.svvs.SFSBFirst;

public class Test {
 public static void main(String[] args) throws NamingException {
     InitialContext ic = new InitialContext();
 
     SFSBFirst ssb = (SFSBFirst) ic.lookup("SFSBFirstBean/remote");
 
     ssb.call("Hello World!");
     ssb.remove();
 }

}

Results I used JBoss 4.2.2 GA on my tests, the first run I did not removed explicitly the injected bean. Here's the output:

01:09:11,089 INFO  [STDOUT] br.com.svvs.SFSBFirst: called(Hello World!)
01:09:11,142 INFO  [STDOUT] br.com.svvs.SFSBSecond: called(Hello World!)
01:09:11,161 INFO  [STDOUT] br.com.svvs.SFSBFirst: removing(Hello World!)
01:09:11,162 INFO  [STDOUT] br.com.svvs.SFSBFirst: destroying(Hello World!)

As you can see SFSBSecond was not removed. So if I want to remove it, I have to explicitly call remove() on the second bean during the remove() of the first bean. The second run was made calling remove():

01:11:51,702 INFO  [STDOUT] br.com.svvs.SFSBFirst: called(Hello World!)
01:11:51,706 INFO  [STDOUT] br.com.svvs.SFSBSecond: called(Hello World!)
01:11:51,714 INFO  [STDOUT] br.com.svvs.SFSBFirst: removing(Hello World!)
01:11:51,719 INFO  [STDOUT] br.com.svvs.SFSBSecond: removing(Hello World!)
01:11:51,721 INFO  [STDOUT] br.com.svvs.SFSBSecond: destroying(Hello World!)
01:11:51,721 INFO  [STDOUT] br.com.svvs.SFSBSecond: destroying(Hello World!)
01:11:51,721 INFO  [STDOUT] br.com.svvs.SFSBFirst: destroying(Hello World!)

This time SFSBSecond was removed, but I am not sure why it appears to be called two times, probably a JBoss bug. Since a second opinion is always good let's see how the glassfish v2 application server behaves. The first test was not calling remove() on SFSBSecond:

[#|2008-10-29T02:37:08.263-0200|INFO|br.com.svvs.SFSBFirst: called(Hello World!)|#]
[#|2008-10-29T02:37:08.264-0200|INFO|br.com.svvs.SFSBSecond: called(Hello World!)|#]
[#|2008-10-29T02:37:08.267-0200|INFO|br.com.svvs.SFSBFirst: removing(Hello World!)|#]
[#|2008-10-29T02:37:08.267-0200|INFO|br.com.svvs.SFSBFirst: destroying(Hello World!)|#]

Well, as the JBoss server Glassfish doesn´t remove the inject SFSBSecond bean. And now calling remove() explicitly:

[#|2008-10-29T01:50:28.722-0200|INFO|br.com.svvs.SFSBFirst: called(Hello World!)|#]
[#|2008-10-29T01:50:28.722-0200|INFO|br.com.svvs.SFSBSecond: called(Hello World!)|#]
[#|2008-10-29T01:50:28.725-0200|INFO|br.com.svvs.SFSBFirst: removing(Hello World!)|#]
[#|2008-10-29T01:50:28.726-0200|INFO|br.com.svvs.SFSBSecond: removing(Hello World!)|#]
[#|2008-10-29T01:50:28.726-0200|INFO|br.com.svvs.SFSBSecond: destroying(Hello World!)|#]
[#|2008-10-29T01:50:28.726-0200|INFO|br.com.svvs.SFSBFirst: destroying(Hello World!)|#]

That appears to be right, the bean was removed in the right order and all variables still in memory. Conclusion Explicitly remove the injected beans, the application servers JBoss and Glassfish will not do it for you. I think that´s kind odd, since they injected the Bean and a Stateful bean can have only one client it must be destroyed when the client is destroyed, but that´s the way things are.

NFS concurrency on distributed systems

2008-07-19T18:04:00.000-07:00

Last week I had a hard time trying to figure out one way to scale one application to many machines and many instances on each machine using one NFS server as point of access. I believe I found one solution to this problem and here I will explain some of what was done. First let me explain the problem more clearly, these are the requirements: * A arbitrary number of files will be on a exported NFS file system. One * consumer application have to read a file and import it contents to a database. * Each file should be imported only once. After import the file should be moved * to a archive directory. No file can be ignored or skipped if contains valid * data. One or more consumers will run on one machine. One or more machines * will run on the network. We can state this problem as a I/O concurrency problem but robust solutions as Chubby or similar I cannot use because some constrains. These are the constrains: * I have not access to the NFS machine except by the exported file system. * The solution should be a simple refactoring of the current application, not a major project nor a redesign. * The application is written in Perl. On a first thought I went search for file locking solutions, the idea was simple. Each consumer will lock the file before attempt to read it, other consumers should skip the file if it's locked. NFS has a problem with locking as you can read "Use of NFS Considered Harmful", to get things worse I was told that the NFS Lock Daemon will not run on the NFS server. Digging more on file locking you will find this on Linux open() manual:

O_EXCL is not supported on NFSv2 or on Linux before kernel 2.6; it is supported
on Linux 2.6 and later, with NFSv3 or later. In environments where NFS O_EXCL
support is not provided, programs that rely on it for performing locking tasks
will contain a race condition.  Portable programs that want to perform atomic
file locking using a lockfile, and need to avoid reliance on NFS support for
O_EXCL, can create a unique file on the same file system (e.g., incorporating
hostname and PID), and use link(2) to make a link to the lockfile. If link(2)
returns 0, the lock is successful. Otherwise, use stat(2) on the unique file to
check if its link count has increased to 2, in which case the lock is also
successful.

So, trying to acquire a lock exclusive on a NFS file system will fail and despite my best efforts the solution given not worked on my tests. After more researching I found the CPAN module File::NFSLock. Some testing was done and the module worked fine on concurrent processes on one machine. All processes respected the lock and the files were read it without problems, but as you might expected things broken on multiple machines. Another module was tested, File::LockSimple, this module as a parameter for NFS file system, but still fails on a multiple machines concurrency, as File::NFSLock worked fine on one machine concurrency. What were the problems found? Well, the most common problem was that multiple consumers failed to move the file to our archive directory. The odd thing was the consumers thought that had the lock, but that was not true. Somehow two or more consumers had all confirmations of a lock achieved but the move operation failed. As a example I will explore File::LockSimple code when it checks if had the correct lock: -- snip --

# Attempt to create lock
if (open(FILE, ">$lockfile")) {
    local $\ = undef;
    print FILE "$stamp\n";
    close FILE;
    open(FILE, $lockfile);    # Check lock
    my $l;
    chop($l = );
    $locked = $l eq $stamp;
    $l = ;            # Must be EOF
    $locked = 0 if defined $l;
    close FILE;
    last if $locked;        # Lock seems to be ours
}

-- snip -- The idea is: * Write a unique stamp to our lock file. Open the file again and read if the * stamp is still our stamp. If it is, we have the lock, if not, we loosed the * race. After some testing I've found that when multiple machines race for the lock sometimes there's no winner, and the lock stays without parent and the file is skipped by any other consumer, I had to go with another approach. The solution One premise on file locking is that some other process will try to read/write to a file when other process is using. The solution basically is put the process in some wait state until the owner of the lock releases it. In this line of thought we have one precious requirement, the file should be in a known place and you cannot move it. But this is not a problem in my requirement, I can move the file away so the solution depends on that. The algorithm in plain text: "Create a list of all files in the source directory, create a unique directory path, for each file in the list test if file still exists on the source directory, if not skip this file and go the next file. If file exists in source directory try to move it to your unique directory, if move fails skip this file, if not test file the file is in our unique directory, if it is we have the lock, if not skip this file and go to the next file." The premise of this algorithm is: "No file could be moved from one place to two other places at the same time." This should hold true for any number of requests to the file system. Here is the algorithm in pseudo code:

source_dir = "some directory"

f[] = list_of_files(source_dir)

uniq_dir = create_uniq_directory()
 
for n=0 where n <= size_of(f[]) do {         
    if(file_exists("source_dir/f[n]") == true) {
        if(move_file("source_dir/f[n]","uniq_dir/f[n]") == true) {
            if(file_exists("uniq_dir/f[n]") {
            # We have the lock
            }
        }
    }
}

Let's explore the algorithm, first I create a list of all files on my source directory, the real number can be about 20,000 files. Then a unique directory is created, the directory is unique for any number of consumers, I use the name of machine (hostname) + process ID. For each file in our array we ask if file still exists, if not we skip because some other consumer probably already got the file. Now we try move the file away to our unique path. Pay attention that we try to move the file, because after our first test the move can fail if some other consumer move it first. Seems odd, but believe in me this happens. If the move returned true we go a test if the file is in our unique path. The thing is, move() can return true even if failed. So we test it again to see if is telling the truth. Why move() return true even if failed? The catch is in NFS system, it has a cache feature and could return true to the caller and then pass call to the kernel of the server system. What happen is that two clients could issue a move() request, NFS responds that OK because think that the file is still in the source directory. But when calls the kernel it detects the absence of the file and fail, but the client will not know what happened. I'm not sure if this is the real case or how NFS handle such calls, my tests point that something like I described above occur. The trick is in two things. First, the unique path. Since is unique (directory + file) NFS will not have cached information about it. So when I ask "file exists in my unique directory?" NFS has to check the path. The second, the move operation is in fact a rename() operation and a rename operation is a atomic. This way we guarantee that when two calls arrive to rename one file, one of the calls have to fail. So in the same machine, in a unknown other one call will fail:

rename(file, file_new_name);
rename(file, file_other_name);

The final implementation Here is a simplified version of what was implemented:

#!/usr/bin/perl

use strict;
use warnings;
use File::Copy;
use Sys::Hostname;

use constant SPOOL_DIR => "/tmp";

my @files; # get files from some object...

# Create unique spool path (concurrency algorithm)
my $file_spool_dir   =  join('/', SPOOL_DIR, hostname, $$);
if(! -d $file_spool_dir) {
    mkpath($file_spool_dir) or die(" ** Could not create spool directory ($!)");
} elsif(! -w $file_spool_dir or ! -r $file_spool_dir) {
  die(" ** Cannot write/read spool dir check permission ($!)");
}

FILE: foreach my $file_path (@files) {

  next FILE if ! -e $file_path; # File still exists?

  my $file_name = File::Spec->splitpath($file_path);      

  my $file_spool_path = join('/', $file_spool_dir, $file_name);

  # We try move our file to a unique path where nobody will race with us.
  if(move($file_path, $file_spool_path)) {

     # Check if file was moved to our uniq path;
     if(-e $file_spool_path) {
        # we have the file!
     }
  }
} # end of foreach file.

I have to make a note, using File::Copy method move() I'm in fact renaming the file, the documentation of File::Copy states:

If possible, move() will simply rename the file.

As mentioned before, the rename operation is atomic so we espect that move() will not break our first premise. Final considerations I tried to make the algorithm the best I could, formal proof is hard especially in concurrent and distributed systems. But the process worked fine on 25,000 files acessed by 20 consumers in two machines. No error, no file was lost or skipped without a good reason. Despite I can't say this is a bullet proof algorithm, by the tests it seems hold well.

Using JQuery to create dynamic HTML select elements.

2008-07-07T18:22:00.000-07:00

This week I'm working a lot with JQuery and when searching for handling html select element I found little information about it, so I decide blog a little about this subject.

Typically you want add and remove options from a select box, so I setup a little page that does an append to a select and removes an option too.

We will have a Fruit Basket, we put a item in our hand and then eat it. Here's the basket (listing 1):

1 <html>
2   <head>
3     <script type="text/javascript" src="jquery.js"></script>
4   </head>
5   <body>
6     <table>
7        <tr>
8          <td>
9            Fruit Basket<br>
10            <select name="basket" size=5 >
11              <option value="apple">Apple</option>
12              <option value="banana">Banana</option>
13              <option value="orange">Orange</option>
14              <option value="grape">Grape</option>
15              <option value="starfruit">Star Fruit</option>
16            </select><br>
17            <input type="button" value="Pickup" name="pickup">
18          </td>
19          <td>
20            Hand<br>
21            <select name="hand" size=5 ></select><br>
22            <input type="button" value="Eat!" name="eat">
23          </td>
24        </tr>
25     </table>
26   </body>
27 </html>

This HTML is a simple one, the table was inserted just to put the selects side-by-side. Note that I didn't put any javascript event handler like onClick(), onChange() and etc. This because with JQuery you can add the event handler very easily without cluttering the HTML.

Now comes the good stuff, insert a <script type="text/javascript"> tag and let's see JQuery's magic (listing 2).

1 <script type="text/javascript">
2 $(document).ready(function() {
3   // the good stuff goes in here...
4
5   // when somebody clicks on Pickup...
6   $("input[name='pickup']").click(function() {
7
8     // there's something selected in our basket?
9     if($("select[name='basket'] :selected").length > 0) {
10
11       fruit_text   = $("select[name='basket'] :selected").text();
12       fruit_value  = $("select[name='basket'] :selected").attr("value");
13
14       // put the fruit in hand...
15       $("select[name='hand']").append(new Option(fruit_text, fruit_value));
16
17       // remove the fruit from the basket...
18       $("select[name='basket'] :selected").remove();
19
20     } else { return false; } // don't bother with a empty selection
21
22   });
23
24   // let's eat!
25   $("input[name='eat']").click(function() {
26       $("select[name='hand'] :selected").remove();
27   });
28
29 });
30 </script>

Now let's explain what is happening.

If you already knows JQuery you recognize the line 2, this is a simple way to trigger your script when the document is ready, it's a good practice. Insures that your script will not fire before the browser finished rendering all page elements.

In line 6 and 25 we define one anonymous (formally a lambda) function that will be fired on a click event for our both buttons. Pay attention on JQuery selector syntax it's very important! In line 6 we say to JQuery:

$("input[name='pickup']")

and JQuery understand this:

"Give me the input element which has the attribute name equals to pickup."

If we were ambiguous in our selector JQuery will return more than one element, in this case we could iterate over the matches found with each() helper.

We ask to JQuery in line 9 if there's something selected in our basket, if the length of selected fruits is greater than zero let's do something.

We use some accessors from JQuery and save the text and the value of attribute "value" in two variables, lines 11 and 12. After that, in line 15, we tell JQuery to append a new Option object which is basically the same option selected in our basket.

Now the last step in this event is remove the fruit from the basket, one way to do this is calling remove() on the JQuery object.

If you payed attention on the above explanation lines 25 to 27 should be clear. We select what is selected on our hand and just "eat it" (in fact we remove the element). That's it, simple and clean.

Easy (lost all your data) Profile Transfer Tool for Windows

2008-07-06T14:06:00.000-07:00

Recently in my company we had to change domains in Active Directory so all users couldlog into the corporate domain. Well, I previously had a experience like that and I've lost all my configurationdata. But we now used the "Easy User Data Migration" (I really don't remember the correct name) tool fromMicrosoft.

Guess what? I loosed all data again! But I managed to get it back with a simple action, but I was lucky inside the company, others had to search file by file.

First how Microsoft store users data? In a shallow view you can look at your directory:

 C:\Documents and Settings\username.DOMAINNAME

So, knowing that the transfer would not work I used the backup tool and stored all in my USB drive. After thatI was recommended by the administrators to use the Easy Transfer Tool. Ok, I did it, and for my surprise the tool gathered information about my C:\perl, C:\ruby and others directories that I think there's nothing with my profile.

When the tool finished it stored about 760Mb of information in my transfer directory. I thought "It's ok, this is about the size of my C:\Documents and Settings\username.DOAMINNAME dir, so it's alright". Oh sooo naive!

The domain of the computer was changed and I used to tool to recover all my data, let's talk about my data.

First, my data is a bunch of profile variables that I have to define for Java, Oracle and other personal applications. Is my Firefox profile, my Thunderbird profile, my Twirl profile, Adobe air, Launchy shortcuts, SVN and unix tools for windows. The problem with these data is that I'm constantly defining new variables, shortcuts and getting extensions because any programmer don't like to type the same thing twice when it can be done by a script.

My Desktop is, as for many programmers, full of little tweaks so works better and fast. I had "Process Explorer" launched by default (any programmer in a Windows box should do this) and minimized, well a lot of little stuff.

So I was not willing to tweak the system all again. So I logged in the new domain and launched the tool, waited, waited and message was Your data was recovered, some you should do manually, Uh Oh!

When the desktop came alive the first thing I checked was the profile variables… and …. none! My Firefox forgot about me, and the others applications too. So, what the hell was transfered?! I think just the Domain Stuff, just things I assume no one should worry, even know about it. But about the stuff that was important to me? Even the Desktop Wallpaper was not transfered!

Well, are you asking, how did you recovered your settings? Just renaming the directory:

From:

C:\Documents and Setting\username.DOMAINNAME

to:

C:\Documents and Settings\username.NEWDOMAINNAME

Bit there's a catch, I'm the Administrator of my machine so I can do that, and the second step was change theowner of all files to my username at my new domain, and only administrators can do that.

What bugs my a lot is why the tool archived all files but fail to put them back in place? We are not using a hack like a samba server or a ldap server mocking a windows server, we did it by the book, but still Windows failed to provide a good experience and took about three days to some users to put the Desktop in order, as I said, I was lucky.

That's it. If you think that a simple change in domain is a simple thing, think again, back up your data and be ready to loose your settings.

Is Selenium testing good or bad?

2008-07-03T21:52:00.000-07:00

I've just read this post of Jay Fields about Selenium. I have some different points about Selenium testing and its use, I'm owning a post about Selenium so the time came. First of all, I 'm not a experienced Selenium tester, I've heard about it and waited until have some time to put the tests on a legacy application that I support. Let's put on context, Selenium for me was hard to learn. Jay says that is a easy to learn but wasn't for me. Couple of reasons: The Selenium IDE is easy but limited, so I went for the big fish SeleniumRC. My legacy application is very bad implemented. About all pages has invalid HTML syntax and JavaScript errors, and the application has a lot of pages and interactions. As Jay points out, Selenium has a lot of languages you can write tests and we can get a little confused about what language to choose to write them. This can be a problem but I don't think so, if you think a little why you should choose a language in favor of another. You have just to make the decision of what language suites you better, can be the language you are most familiar or, as I decided, the language of the implementation of your application. I've chosen the second one because my team writes in Java, my app was written in Java and i think is a good thing to keep your tests with your application in the repository. Java has other benefits, typing, with Perl I had to check all return types and look at the API all the time. But with some practice you get how Selenium works and you get to write tests more rapidly. I really don't know about other experiences with Selenium as Jay mention, if developers had to throw their tests away after certain size, this still is not a problem for me, maybe in the future Selenium bites me, still... Jay argues that Selenium tests are slow and browsers are buggy, it is absolute true. A simple suite will run in 10 minutes or so, and that's slow. But I don't think this is a Selenium problem, in fact Selenium is fast where it can, surely more fast than written down scripts for users to test the application. Browsers. Oh! browsers... we developers still fell a major pain when someone shouts "It doesn't work for me!" The browser war made a lot of collateral damage, but still it's not Selenium fault. In fact Selenium came to help us with these damages. Being capable of programatically test my application on a Firefox and IE is a bless. When a test fail little information is provided, as Jay puts, and it's true. I agree that if more information was provided will be a big gain, but this is not, as far as i know, Selenium's fault. We have to remember that we are testing the final product, the one which is stripped of "business logic" (i know that the behavior is business logic but I'm talking about of the business logic we hid in our classes), in this case we're experiencing the user environment and it's a harsh one. I develop for the web for about nine years and some problems in the user environment are really bizarre. In my opinion Selenium is a excellent initiative and has much value, of course that is not the only testing you should do. When using Selenium you are able to catch some very complex interactions that a simple unit testing will not, by the fact that "in pieces" the application behaves right but when the user go from one state to another and back again some missing parameter, a misplaced element on a page and etc, can bite you. For who wrote such tests using LWP library in Perl and looking for elements using DOM parsers I think Selenium is great. Can be better? Of course! I had a difficult time in find complete examples and I think that the docs still need improvement but I've found my way through these and the faulty, sloppy made application can be tested and refactored without noticing a break after weeks of development, and that's good for me. I can work more confident on this application then before, this is a good sign.

It's been a long time from my last post...

2008-07-01T19:54:00.000-07:00

... so let's talk: Are you smart?

In fact, I just read this post (lengthy post) by Steve Yegge and the funny thing was that I were just thinking about how not smart I am. (Note: Blogger shortcuts Ctrl+I,B,... not work on Safari on a Mac?). Watching his talk at Google I/O 2008 got me thinking "Uau, I'm really a low level developer", then I went to his blog and saw the article "Done, and Gets Things Smart".

In fact this is not a really bad felling, you know, if you get too confident you make mistakes, thats the way things are. It's the same when you judge your skills at some field, as a developer who always searches about the trends in my field I face the overwhelming amount of information that I don't have every day. Languages, frameworks, things that were good patterns and now are listed at anti-patterns, new development tools, new debug techniques, new testing frameworks and so on.

- I should make a parenthesis here about patterns.

The overuse of about anything creates a bad thing, like your plate at lunch. Nutritionists says the tonal variance of your plate is a good sign. If fill up your code with patterns this is NOT a good sign. Patterns have a defined problem they try to solve, if you fill your code with patterns, after patterns, after patterns, in a simple view, you have problems, and problems, and problems at your code. A good way to approach a pattern use is: "Oh! How I gonna solve this problem? Maybe someone thought about it and came with a solution." And not "Oh! What patterns I should use to build my application?".

- Back to the main subject.

After read hundreds of blog posts every day, articles, watch talks and so on, you should not (as I try too) get depressed. There two main reasons I can think of, the first is that your field is growing fast, this means jobs, money and space for everyone how likes technology. The second reason is that you are trying to keep with the wave, you are searching for improvement and surely you'll be a better professional tomorrow than you are today. Of course a filter should be applied in the enormous amount of info, but we learn how to choose the right information with time.

The message is: Be smart in not being smart, do the things you know, what you don't know go outside and learn it. As the greek philosopher Socrates said "I know nothing except the fact of my ignorance" and the oracle of Delphi stated that he was the wisest man.

[]'s and keepcoding.

One question worth asking in a interview.

2008-03-06T17:44:00.000-08:00

There's a lot references how to do a I.T. interviews, so of course will spend you of more about this topic except by one little thing, a little question that is, for me, a good one to fell how comfortable is the candidate with the technology of the job.

Say Java, ask something like: "What feature of Java annoy you?" or "What feature of Java that you don't like much and could be improved?".

The point here is very simple, is easy to say the advantages but requires a lot more know how to no the problems. The answer don't have to be perfect, sometimes we think that one thing is annoying without knowing that is the best. To change opinions is part of growth it's natural, but have a opinion is more important.

Answer this question is harder for someone that never thought how improve their processes, or took a technology as a final solution, or another things that annoys me as well.

I.T. methodologies and religious paths.

2008-02-15T18:51:00.000-08:00

Is not a big news to know that I.T. companies have problems with methodologies. They want them but for some reason they fail implement them, of course this don't prevent a big logo with CMMi, ITIL, RUP, XP, Scrum, ICONIX and etc to popup in your face. Jut hire some experts to give some presentations, put some docs and fancy names for these and you are ready.

The impression I got is that competition and "everybody says is great" is the most compelling reason for this kind of behavior. A company should improve always so get some methodology is a improvement, get two is more improvement, mix the best of the two is better! Well, not.

One big company said to me that they have a "kind of CMMi" they extracted some steps and used only what was the best about CMMi. What is the problem with get the best of? The problem is that there's no such thing as the best part of a methodology, you can't decide this, you shouldn't decide this.

Take religion, have you heard some people saying that they are Spiritualist, Catholic, Buddhist, Muslin and so on? That they think that some concepts of one religion are great but others not. Ok, everybody should analyze, think about, take conclusions but they have to remember that not all aspects are in our comfortable zone. If you want to call your self a Buddhist you should remember that implies not to drink beer. And this is not a silly rule that some monks thought about, there's a reason for it and sometimes you can't see the point.

What I trying to say is that, companies are very naive thinking that they can point out what to adopt or not. The CMMi, as a example, is well thought and every aspect is there for some reason. When you try to decide what is the best for you is it certain that your comfort zone will drag you to rules and procedures that favors your zone, that's the big and real problem.

Do not try to mix to make it best just stick with one, if is not working try another, and another, but the mix don't improve your results, in fact can lead you to a false perception that methodologies do not work. I know that there's no perfect solution and after you have a methodology then you can tweak it around to fit your necessities.

Keep in mind that: "Simplicity is the shortest path between two points" - Bruce Lee.

7 Things a developer should know to survive big companies

2008-01-14T20:26:00.000-08:00

Not every company will embrace new technologies or methodologies, but you probably will be in charge to achieve a similar performance as companies that adopt new technologies and methodologies such as Scrum and Rails. So, what you should know to get things done in a agile way but without a formal agile method inside your company? Here's some things that can save your time and health. Know what is: Paradox of Choice Users are not evil or dumb, they are humans. And humans who knows the problem you want to solve, even if they are evil or dumb, that's a fact. If you have to deal with users try to not block your mind to their presence, they have the key of your success. If you block them, then you have a good chance to deliver something that would not solve their problem. Software is all about users, right? When talking to users always, I mean always, be aware of the Paradox of Choice. In companies that use the old "Get all requirements -> Some stuff -> Put in production" flux, you probably will ask if the user want this or that, and as influenced by the Paradox of Choice he or she will want all you offer and more. You know were this ends, in a big multi-function software that forgets to solve the main problem in a good way. Know this: If you can't model you can't build it. This quote is from John Walker, and you should be aware of that. You should know ways to model your problem in a more formal way, maybe using UML. The main thing here is to abstract a little before code anything, some simple diagrams can save your life, you can avoid wrong architectural choices if you draw your idea in a paper before start coding or writing a specification. You don't have to master UML or Canonical Abstract Prototyping , but you have to know the enough to represent your mind outside it. This helps to organize your thought and maybe linguistic determinism plays a hole in here. The constrains of a UML could save you from some traps, as the diagram gets confuse and clouded maybe you have to think a little more about the problem. Know this trap: Analysis Paralysis When you have too much worries in modeling you can be bitten by the "Analysis Paralysis" anti-pattern. As I said before you should model, modeling is good, modeling too much not. Some insecurity about the project can make you afraid to deliver your specifications until they are perfect. But what you have to keep in mind is that there's no such thing as a perfect specification. In fact I think that "specification" is not a good name because it's more like technical guidelines than a complete solution. "Working Software Is Your Primary Goal", Scott W. Ambler wrote in "Agile Modeling". That's it. If you try to model, or specify a perfect document you got it wrong, your final product is not a model. Note that a abstraction cannot be in same details of what is abstracted, if so, there's no abstraction at all. Experience will teach what is a good enough model. Know what is: The KISS principle Keep It Simple, Stupid. Some problems are complex, but shouldn't make them more complex. When trying to solve a problem try to be simple about it. In software you can design a solution that will take hundreds of lines of code, a bunch of classes and maybe solve the problem, that can be a problem. Software has this "soft" nature and people think if the goal is achieved then the solution is right, not quite. I remember in my Linear Algebra course in college when we got the right answers by the wrong methods the professor marked the question as wrong. "You answered right but you solved it wrong." that's harsh, but true. When you try to solve a problem think carefully how you are solving it, getting the right answer is not a indication that you solved it right. (see this too) Occams Razor is a good way to think about simple solutions, if is getting too complex, if the code start to smell, think about it again. I'm not saying that every problem has a simple solution, but when getting a to complex solution, make a review to be sure that your path is the more simple, even if it's a complex path. Know the: Scrums and XPs principles As the introduction of this article said you can't use Scrum or XP in your company, but this is not excuse to not know Scrums and XPs principles and foundations. Scrum and XP are methodologies that try to solve common problems in software development and some principles are applicable even if you can't use it explicitly. Like the Scrum sprint, we can't ask the user to always refine priorities, but we can try to deliver a functional piece of the software every week or so. User stories of XP is a good way to organize requirements. Using some of the concepts and knowing the problems they try to solve can help you and your team to reach better results. Know what is: Function Point Analysis Function Point Analysis (FPA) it's a metric for software. FPA is hard to do, takes time and if is not a procedure in your company you probably wont be granted the time to do it. But there's something about FPA that is very useful, the "User Perspective". FPA is all about how the user sees the system, every function the system is capable to do and the your knows about is subject to count. The key thing is, if the user asks for a "sort by name" in a column this will affect the complexity of your project. So, before you say "yes" to something that seems simple, remember that even a little "sort by" adds cost to your project. You can make your user aware of this and construct a good relationship before start to promise things that you can't deliver. When you use FPA you get a notion of the impact of "little" things added, so even if you can't use a formal FPA approach try to make a little informal count (but don't cheat in the count) to see more clearly how "little" functions scale to big efforts. Really know about: Human-Computer Interaction That's tricky, but users need help to choose a interface that suits them. I've seem developers leave at the user the responsibility to transform a complex spreadsheet system in a new interface for the web, the "draw what you want" approach. Worse, one developer just copied the spreadsheet interface to a web interface. That's not the way to do it. Users are not aware of the possibilities of a alternative interface, and the developer who copy a interface probably is not helping the user at all, sometimes the propose of a new system is to avoid spreadsheet interface in the first place. User generally has a idea of what he/she wants of the interface, some want fast input, some fast search, some want most info in one screen and others want a step by step, like wizards, to complete a process. When you get this information you can draw screens that help the user but, do not let the user freely draw the screen, use a guideline to help you and the user, like Apple Human Interface Guidelines. Is not that you should write Apple like softwares but when you are familiar with some of the concepts, you get a better idea if a interface will work or not. Who never designed a interface which users keep making the same mistakes over and over again can throw the first stone. When this happens is a wrong designed interface not a dumb user. If your users can't use your software you are doomed, doesn't matter if you have the finest code that a programmer can ever produce. You be in a sad place where your users keep using a software that you think is a crap and not using your new top designed solution. Usability is something you should worry about, not knowing could kill your project. That's it

Want a cake? My first question is...

2007-12-12T17:46:00.000-08:00

...what should be the size of this cake? I've began some Function Point Analysis (FPA) with a new project in my work, so far this is what I got, but first let's talk about my cake. The project is not a very complicated one, is more a store and retrieve information of some sort. Since I'm working at a bank is basically numbers about a client and without much calculation, so it's a simple project, nothing better to exercise my sizing methodology skills. First impression: Function Point Analysis it's simple to understand. Note that I've wrote to understand and not only it's simple, this is because I managed to get a function count but the implications of this number is still unknown to me. I've looked at my SRS document and applied the FPA rules and got a 354 FPs, this is greater than the example in the book I'm using, and the example seemed more complex at first. So I'm still a little puzzled. Using a very simple extrapolation, a productivity of 10 FP People per month and a team with 5 people, I got a 7 months project. This is feasible? Thinking a little about it this is feasible for me, if we're talking of the completed project, start to end. But a simple project with 7 months to get it done? Yes, here some reasons:

The project will be outsourced.
There's no clear methodology in use, RUP, Agile or else.
It's not a single software, in fact is a extension for a web tool in production, requires integration.
It's a global tool two teams of different countries should work together.

Even a simple project can suffer a lot in these conditions, so 7 months it's not a surprise, but my FPA is right? Despite the presence of GSC in a FPA to weight the environment requirements in a project it only count from the user view of the project, but my worries are about things that a user cannot see. Well, my count is wrong? I'm sandboxing the schedule of the project? If the technical difficulties that worries me were removed the count would not change, what about the schedule? Only time will tell, but I keep in mind that there's no silver bullet.

If programmers have make a plane

2007-12-04T17:43:00.000-08:00

You probably felted like this before.

Two Screens Laptop

2007-11-10T20:31:00.000-08:00

Couple days ago I was playing with a simple GPS from my mother and I
realized that in future, not so far away, touch-screens probably will
be ubiquitous as mouses are. Well, in fact I discovered that a lot of people are working on this and I have too some thoughts about how computers will interact with us in future.

No all talking interface ("Computer do this, do that..."), but certainly touch-screens all around (see this too). You can see some movement in this direction with the new gadgets of Apple, the iPhone and iPod Touch.

Its fantastic how the desktop manipulation is getting better, be amazed with BumpTop, now think about that kind of behavior with a touch-screen. Since my hypothesis is that will be only touch-screens lets see how a notebook will look in the future, this is a two screens laptop (a rough example below).

Note that we can think in a computer that will resemble a todays laptop but with a crucial difference, there is no keyboard. To get the idea think in one computer with another screen in the place of the keyboard. You can slide your finger and a "virtual keyboard" will appear and you type normally, another gesture and the keyboard goes away.

As iPhone and iPod Touch, if you turn the computer like a book the screen turns too and the pages of your document takes both sides of the laptop and you can turn pages with a finger slide.

This sounds like document oriented only, but how about applications? Well, see this two demonstrations of XGL + Linux (here and here) and think about how this will work in a two screens laptop.

With a two screens laptop we can use it like a real paper sheet notebook, you can turn it, get a pen and draw some diagrams, not write texts, I think keyboards are more useful for writing than pens on a touch-screen, but is possible too.

The number of possibilities kind explode with a two screens laptop, some fancy controls can pop in the under screen like "virtual knobs" to raise the volume of your music, some very fun games and any stuff you can think of, making the desktop metaphor (after decades) more closely to the real desktop but very powerful.

Sabe aquele software?

2007-10-30T19:29:00.000-07:00

Pois é outro dia tive que lembrar de um software que utilizei para modelar tabelas no ORACLE e o nome do dito escapou da mente, felizmente consegui reproduzir uma busca e então o encontrei. Foi nessa ocasião que percebi precisava criar um pequeno catálogo dos softwares que já utilizei, pois a memória já não está as melhores. Devo estar acumulando uns onze anos em frente a um computador, e já passei pelo Netscape Gold (lembram-se dele?) até Mac OS 10.4 (meu OS atual). Na verdade não é apenas manter um histórico mas sim uma tabela de referência, já que alguns softwares foram muito úteis e realmente merecem uma propaganda extra. Graças ao Google a tabela pode ser acessada aqui, e prometo atualizá-la sempre.

Vim - Facilitando tarefas repetitivas (de noweb para PDF)

2007-10-06T17:19:00.000-07:00

Eu sou um notório adepto do editor Vim, carrego até uma versão portável em minha Pen Drive. Neste post vou mostrar como podemos automatizar algumas tarefas repetitivas, neste caso em particular a compilação de um documento noweb. Se você não está familiarizado com o conceito de Literate Programming, CWEB e LaTeX não precisa se preocupar pois estes conceitos não são o foco deste post.

E de qual tarefa repetitiva estamos falando? Bom, eis os comandos que eu tenho que executar para obter um documento em formato PDF a partir de um documento noweb.

Primeiro executar o noweb para criar um documento LaTeX:

noweave -delay HelloWorld.nw > HelloWorld.tex

Executar o LaTeX para produzir um documento DVI:

latex HelloWorld.tex

Converter o documento DVI para PDF

dvipdf HelloWorld.dvi HelloWorld.pdf

Abrir o documento PDF para verificá-lo:

open* HelloWorld.pdf

* este comando pertence ao Mac OS X, para outros sistemas você deverá alterar para o que for mais apropriado, por exemplo no Linux podemos utilizar acroread HelloWorld.pdf.

Como você deve ter percebido repetir isso dezenas de vezes enquanto escreve um documento é bastante entediante, e como todo programador é, ou deveria ser, preguiçoso resolvi automatizar essa tarefa recorrente.

Dentro do Vim é possível definir novos comandos e estes comandos podem chamar funções e é isso que faremos. Nosso novo comando Npdf irá fazer tudo isso para tal precisamos defini-lo no arquivo vimrc (se você não sabe o que o vimrc significa comece por aqui) juntamente com uma a função que irá cuidar de nosso trabalho sujo.

Meu Vim não reconhece a extensão .nw, então primeiro vamos dizer a ele que estes arquivos deverão ser interpretados como cweb. Para isso no vimrc escreva:


" associa arquivos .nw ao tipo cweb
au BufRead,BufNewFile *.nw set filetype=cweb

Isso fará o Vim atribuir a sintaxe cweb para arquivos noweb. Agora faremos que, se acaso estamos num arquivo cweb, o Vim carregue comandos específicos para este tipo de arquivo, isso ajuda a separar um pouco os comandos por tipo de arquivo, assim podemos ter comandos com nomes iguais para arquivos diferentes. Para instruirmos o Vim a carregar os comandos escrevemos no vimrc:

" Carrega os comandos para arquivos noweb
au FileType cweb call LoadNowebCommands()

Este comando apenas chama a função LoadNowebCommands() e nela definimos qualquer comando para arquivos noweb. Esta função é bem pequena e simples:


"================
" noweb
"
function LoadNowebCommands()

    " comando que compila um noweb em PDF.
com! -nargs=0 Npdf call Npdf(expand("%"))

endfunction

Dentro dela definimos, neste caso, apenas um comando chamado Npdf e o que ele faz é chamar uma função de mesmo nome que recebe como argumento o nome do arquivo que estamos editando. No Vim comandos e funções ocupam namespaces diferentes e por essa razão podemos definir ambos com o mesmo nome. A linha acima ainda menciona que este comando não aceitará argumentos (-nargs=0).

Até agora o que fizemos é preparar o Vim, primeiro estabelecendo que os arquivos terminados em .nw deverão ser tratados como do tipo cweb. E se um arquivo for deste tipo, então ele deve chamar a função LoadNowebCommands(), e neste caso esta função define um comando chamado Npdf. E este, se executado, deverá chamar a função Npdf() e passar a ela o nome do arquivo atual. Complicado? Espero que não.

Agora devemos definir a tão esperada função Npdf() que irá realmente facilitar nossa vida:


function Npdf(filename)

    " primeiro salvamos o arquivo
 write

 let nowebfile = a:filename " arquivo noweb atual

    " Agora é necessário definir os nomes dos outros arquivos
    " que aparecerão durante o processo.
 let latexfile = substitute(nowebfile, '\.nw','.tex','') " HelloWorld.nw para HelloWorld.tex
 let dvifile   = substitute(nowebfile, '\.nw','.dvi','') " HelloWorld.nw para HelloWorld.dvi
 let psfile    = substitute(nowebfile, '\.nw','.ps','')  " HelloWorld.nw para HelloWorld.ps
 let pdffile   = substitute(nowebfile, '\.nw','.pdf','') " HelloWorld.nw para HelloWorld.pdf

    " executamos o noweave e checamos se houve algum
    " problema reportado por ele, caso sim paramos por aqui.
 exec "!noweave -latex -delay " . nowebfile . " > " . latexfile
 if v:shell_error != ""
         echo "Noweb error\n"
         return
 endif

    " executamos o latex...
 exec "!latex " . latexfile
 if v:shell_error != ""
         echo "LaTeX error\n"
         return
 endif

    " convertemos de dvi para pdf
 exec "!dvipdf " . dvifile . " " . pdffile

    " e agora checamos nosso novo PDF.`
 exec "!open " . pdffile

endfunction

Esta função é bem simples, primeiro salva nosso arquivo e então determina os nomes dos arquivos envolvidos apenas substituindo suas extensões. E então executa os comandos entediantes que queremos evitar. Note que após o comando noweave e latex checamos se houve qualquer erro, se houver devemos parar o processo já que provavelmente não haverá output viável.

Agora quando no Vim podemos apenas digitar :Npdf e ele compila, converte e nos mostra o arquivo PDF final.

Observação: Este post foi atentamente revisado por um aspirante a programador (vide foto):

Observações sobre acesso a banco de dados.

2007-09-25T21:15:00.000-07:00

A pouco iniciei minha incursão mais profundo no mundo Hibernate, e eis algumas impressões sobre tal Pasárgada. Vale também notar que sou recém chegado, estas novas tecnologias podem guardar muitos mais segredos e tesouros do que a mera couraça espinhosa da primeira impressão que recebi. Minhas experiências com bancos de dados nunca foram um mar de rosas, houve tempos em que eu declarava repúdio claro a eles, mas hoje, os reconheço e os admiro em muitos pontos. No entanto todo programador sabe que dentro de pouco tempo em seu projeto, terá que escrever linhas de SQL, e não algumas, mas dezenas, senão centenas. Acabam-se criadas bibliotecas de acesso à banco de dados, no meu caso, essas bibliotecas sempre ficavam enormes. Eis que figuraram-se problemas com esta estratégia, o primeiro e mais óbvio é o alterar de sistema de banco de dados, por exemplo de SQLServer para ORACLE. Apesar de padrão, o SQL não é padrão, por mais paradoxal que isto pareça e portanto alterar de sistema de banco de dados não é trivial. Outro problema é a tonelada de SQLs redundantes e hardcoded nestas bibliotecas, algo bem cansativo e tedioso de fazer. Para solucionar estes problemas alguns criaram estratégias para evitar a redundância e outros resolveram que SQL realmente é desnecessário (em parte). Naturalmente se há um problema é esperado que apareçam "soluções", talvez neste momento a pergunta mais pertinente seja: Há algum problema? Comento isso porque hoje ao ler este artigo tive um sentimento de solidariedade, e isso porque estou em plena guerra com meu cérebro velho e viciado contra as novidades hibernantes. Prometeram-me o paraíso, e eis que escrevi em cima dele "@Entity", mas ele não persistiu. Então comecei a explorar o que eu precisava fazer para que meus objetos sobrevivessem ao próximo apagão elétrico. Descobri, a passos lentos, como persistir meus dados de forma simples e rápida, desde de que eles não estivessem chafurdados em hierarquias complexas e/ou relações "many-to-many". Cada vez mais o modelo orientado objeto não era uma correspondência leal ao modelo relacional, e sabemos que isso nunca foi verdade, estes modelos são diferentes por natureza. Mas, a princípio, as tecnologias para mapeamento Objeto-Relacional tentam nos "poupar" de pensar em como realizar esta correspondência. Fato é, três dias depois ainda não estou satisfeito, e mesmo com os progressos obtidos tenho aquele sentimento, lá no fundo, dizendo que isso deveria ser mais simples (Simple things should be simple). E neste momento que volto a cogitar as boas e velhas bibliotecas de acesso, com SQL hardcoded, diversos métodos quase redundantes e etc. Mas uma coisa elas sabiam fazer bem, guardar os dados. A verdade é que eu desejo apenas salvar certas informações, e mais que isso, salvá-las corretamente quando falamos de modelo relacional, afinal de contas acoplar meu modelo relacional ao meu aplicativo também muito errado, neste caso eu fecharia as portas para qualquer outro acesso. Há a refutação de troca de sistema, afinal de contas uma vez implementado para um sistema de banco de dados, se houver a necessidade de utilizar outro então teremos o trabalho de codificar uma nova biblioteca de acesso, mas nada extraordinariamente complexo. O que me parece, é que devemos ter muito discernimento quanto as tecnologias que vamos utilizar, ora elas são adequadas, ora não. Neste caso em particular creio que a criação de bibliotecas com SQL hardcoded deveria ser a opção mais comum e não o oposto. Tais bibliotecas não são má prática (a não ser que mal codificadas), pelo menos eu não as considero desta forma. Talvez não sejam tão flexíveis, mas flexibilidade demais pode ser estranho.

Formatando código para post em Blogs.

2007-08-17T19:54:00.000-07:00

Eu estava lendo um blog e uma das coisas que me chamaram a atenção no blog foi como o código estava bem formatado. Então procurei na internet uma maneira de formatar o código para posts de blogs. A primeira sugestão foi horrível! Quer dizer que então eu colo o código o Notepad e então executo um Find/Replace para substituir os tabs por espaços? Mas e as cores? E se eu não preceder o código com a tag PRE? Os espaços serão solenemente ignorados. Então alguém nos sugere colar o código no MS Word, nem terminei de ler o post. Lembrei que o Vim possui um conversor para HTML, e realizando uma busca simples encontrei :TOhtml, enfim temos (um pequeno e velho script):

use strict;
use warnings;

my %desc;

$desc{568} = [0,0,2,3,2,4,3,2,2,3,3,2,2,2,0];
$desc{461} = [0,0,2,2,3,4,3,4,3,3,4,2,2,2,0];
$desc{567} = [1,1,0,1,1,3,2,1,2,1,1,1,1,1,0];
$desc{572} = [1,2,2,3,1,2,2,2,1,1,1,0,0,0,0];
$desc{566} = [0,3,2,2,2,2,2,2,1,2,1,0,0,0,0];
$desc{570} = [0,2,2,1,1,1,1,1,2,0,1,0,0,0,0];
$desc{571} = [2,1,1,1,2,4,4,3,2,4,1,1,1,1,0];

foreach my $loja (keys %desc) {

foreach my $day (1 .. 7) {

   my $count = 0;

   foreach my $h (8 .. 22) {

       print "insert into cliente_desconto_passagens
(cliente_id, semana_dia_id, hora, desconto )
values ($loja, $day, $h, $desc{$loja}[$count]); \n";

       $count++;
   }
}
}

O que para mim está de bom e prático o suficiente.

Colocando zeros à esquerda em Java

2007-07-14T02:20:00.000-07:00

Eu sou um velho programador Perl tentando sobreviver no mundo Java, é assim que me sinto às vezes, principalmente hoje que resolvi colocar zeros à esquerda de um número qualquer. O que eu fiz a muito tempo atrás em Perl está desta maneira:

my $contig = '12';
$contig = '0' x (5-length($contig)). $contig;
print "$contig\n";

Ou seja, eu preciso de um número de comprimento fixo de 5 dígitos, e caso este número não tenha este comprimento ele deve ser preenchido com zeros, com certeza essa não foi a melhor maneira de implementar mas isso é um outro problema, eu era jovem. No entanto como implementar zeros à esquerda em Java? Bom eu esperava mais linhas mas não isso (excesso de abstração! e pior, ele usa Java 5 para fazer essa confusão!) e menos ainda passear por aqui sem encontrar exemplos suficientes (mal de javadoc). Felizmente encontrei este site que coloca um pouco de luz sobre o Formatter e a solução ficou como eu gostaria, simples e direta:

private String wrapPageNumber(int page) {
    return String.format("%03d", Integer.valueOf(page));
}

Eu apenas passo o número da página, um inteiro neste caso, e então recebo o número formatado corretamente com os zeros à esquerda. "Simple things should be simple." * OBS: Integer.valueOf poderia ser omitido o que resultaria em (mais simples ainda): return String.format("%03d", page); Mas eu sou do time que possui reservas com o uso de autoboxing.

Complicações e simplificações

2007-07-04T19:28:00.000-07:00

Não é para levantar qualquer guerra religiosa que escrevo este post. No entanto eu gostaria de fazer algumas considerações sobre simplicidade e complexidade. Primeiramente gostaria de lembrar um ditado do mundo Perl "Simple things should be simple, hard things should be possible." (Coisas simples devem ser simples, coisas complicadas devem ser possíveis). Este dizer é, para mim, de extrema importância em diversas áreas mas principalmente no desenvolvimento de software. Fico bastante impressionado quando tenho que atualizar o projeto em que estou envolvido, com Struts e um JBoss, e qualquer mudança, por mais simples, deve ser muito bem feita e levando em consideração todas as variantes destes frameworks. Mais interessante é o movimento POJO, que não passa de um clamor pela simplicidade das coisas simples. A linguagem Java é simples, qualquer programador Java poderia confirmar isso, mas talvez não o faça se ele estiver envolvido num projeto com EJBs, JPA, JTA, JMX, J..., J..., J... Mas a língua é simples a linguagem (aqui digo sobre as especificações) é que é complexa. Não sou um especialista em todas as especificações e problemas endereçados pela J2EE mas tenho a impressão que talvez, somente talvez, algumas coisas poderiam ser mais simples. A alguns anos atrás estive envolvido num projeto de integração, em nosso projeto implementamos comunicação de objetos distribuídos e mais um pequeno framework para a web. Naquela época eu apenas trabalhava com Perl e pouco sabíamos sobre MVC, IOC e Dependency Injection, sabíamos que precisávamos integrar nossas informações, linearizar nossas interfaces e tudo isso com o mínimo de esforço. A boa coisa é que, não foi fácil, mas conseguimos atingir um meio-termo entre novas regras e velhos códigos, e assim nossa framework foi ao ar. Nessa pequena framework caseira as complexidades de separar "Visão" de "Modelo" apareceram e sim, não foram tão simples de resolver, mas mesmo assim parecem mais simples do que as soluções atuais. Mais uma vez, era uma solução caseira, no entanto parece-me que uma solução profissional deveria ser mais simples, mas não foi a realidade. Alguns meses depois de conhecer o Struts fui apresentado ao VRaptor um framework peso leve, e então me senti mais aliviado, pois coisas simples deveriam ser simples. Isso foi ótimo, pois senti-me localizado mais uma vez. Talvez os esforços em tratar as complexidades do mundo corporativo foram muito afundo nos problemas, ou seja, endereçando de maneira robusta estes problemas. O problema é que robustez normalmente vai na contra-mão da simplicidade. E agora o movimento POJO procura manter a robustez no entanto simplificando seu uso, as "anotações" em Java deram uma boa ajuda neste quesito. Agora sinto falta do CPAN (Comprehensive Perl Archive Network) para Java. Já vi iniciativas neste sentido, como CJAN e o JJar. E o que tem o CPAN de tão interessante ao mundo Java? A resposta deve variar, mas minha visão está focada na unicidade de função, ou seja, fazer algo uma única vez e bem. Claro, isso é reuso de código! A grande vantagem do CPAN é a centralização do código, se você pretende fazer algo um pouco mais complicado em Perl antes de abrir seu editor você deve buscar no CPAN, assim é muito provável que alguém já tenha solucionado parte ou totalidade do seu problema. Caso contrário você pode criar seu módulo e publicá-lo no CPAN, mas há algo muito importante nisso. Antes de tentar criar seu próprio módulo há uma recomendação no CPAN, antes de criar seu módulo há um parecido que pode ser melhorado ou estendido? E isso eu acho uma ótima política, força os desenvolvedores uma nova análise do código atual e a colaboração entre si. Enquanto escrevo este post o CPAN informa que há 11830 módulos cadastrados, o que dá um pouco a dimensão da gama de problemas já solucionados, mas onde procurar quando estamos falando de Java? Eis a sugestão, ou melhor, a necessidade que Java enfrenta hoje, uma comunidade como o CPAN. E como Audrey Tang coloca bem "Perl" é a sintaxe e "CPAN" é a linguagem. Já possuímos a sintaxe no mundo Java, agora precisamos criar a linguagem.

Kathy Sierra afasta-se da net devido ameaças

2007-06-27T23:05:00.000-07:00

Hoje em dia estou bem menos sensível, talvez todos nós, é uma pena mas nos adaptamos para sobreviver. Uma avalanche de informações nos atinge todos os dias e grande parte delas, creio que todos concordam, não são boas notícias. Talvez a velha retórica da "Ignorância é uma bênção" volte às nossas mentes, um desejo antigo e ingênuo de nos livrarmos das angústias de nosso moderno mundo. Enfim, a introdução pomposa é apenas para lembrar o que por vezes esquecemos. Mas qual a relação com computação? Bom, tardiamente notei que o blog da Kathy Sierra não era atualizado e fui buscar as razões. Foi uma experiência muito triste descobrir que o blog não era atualizado porque alguém à ameaçava sexualmente e a sua vida. Fiquei estarrecido, como poderia uma pessoa como a Kathy ser intimidada para fora da internet? Para aqueles que não sabem quem Kathy Sierra é pode começar por uma busca no Google, logo vai perceber que ela é um expoente na área da programação e da educação na mesma área. E atualmente estou lendo um livro dela, Head First - EJB. Pelo que pude ler as ameaças foram muito graves e realmente não deixaram outra opção a ela a não ser isolar-se, pelo menos durante algum período. Algumas discussões foram levantadas como a revolta da presença de uma mulher num campo dominado por homens, uma ignorância sem tamanho que nos remete a todo preconceito existente. E qualquer preconceito deve ser combatido viementemente. Só posso lamentar o acontecido, esperar que Kathy se recupere do susto e que possa em um futuro breve voltar a nos informar e ensinar com maestria.

"The Promise, The Limits, The Beauty of Software"

2007-06-10T19:57:00.000-07:00

Essa palestra dada por Grady Booch é ótima, fazia tempo que eu não ouvia sobre software de uma pessoa com tanta experiência e entusiasmo. Creio que não seja uma palestra para iniciantes, pois muitas considerações são observações da experiência, ou seja, padrões e problemas bastante comuns no desenvolvimento de software. Também é uma excelente oportunidade para relembrar o prazer que é ser programador e estar envolvido nesta área tão complexa e tão necessária ao nosso modo de vida.

Todo programador Java ruim que eu conheço...

2007-06-10T14:20:00.000-07:00

Esse tópico foi sugerido no GUJ e das respostas dadas (daquelas que li dentre as várias postadas), a que eu concordo em todos aspectos é do Fabio Kung (professor da Caelum): "Todo programador Java ruim que eu conheço só sabe Java." Mantendo em mente:

"Learn at least one new [programming] language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut."
--- The Pragmatic Programmer

SCJP5 = SCJP5 + 1;

2007-06-05T19:10:00.000-07:00

Eis o resultado de uma verdadeira saga! Enfim consegui obter meu pedigree, agora sou um Programador Java 5 certificado.

Já não basta o stress de uma prova de certificação, tive as seguintes confusões como um bônus:

Uma greve das Lotações; fiquei uma hora preso na Rebouças sem qualquer chance de chegar no horário para fazer a prova. Tive que parar o carro e ir de carona na moto de um grande amigo (Consegui chegar no horário).
Durante a prova houve uma queda na rede e parou tudo! Fiquei 20 minutos esperando para saber o que iria ser das minhas 30 questões respondidas, felizmente o software volta do ponto que parou e trocaram meu computador.
Depois da troca, prossegui com a prova até descobrir que as questões de "Drag and Drop" não funcionavam neste novo computador. Tempo correndo, chamei o técnico e a pessoa responsável pelo teste, e obtive a seguinte resposta: "É verdade alguns alunos reclamaram disso mesmo! É problema no software da Prometric, pule a questão...", perguntei se eu poderia voltar ao outro computador, cujo o software funcionada, e então: "Vixe, não vai dar não, tem outro aluno usando e a prova dele demora umas 4 horas." Prefiro não contar o resto do diálogo, enquanto os meus últimos 20 minutos de teste passavam durante toda a argumentação.
Ao final da prova um comprovante deve ser impresso, foi? Claro que não, tive que esperar o técnico configurar outra impressora. Por fim, entrei às 11:30 para fazer uma prova de 3 horas, sai lá pelas 16:10!
Ah, era meu rodízio, quando cheguei ao carro eu não podia sair, tive que ir ao Shopping Eldorado esperar até as 20:00, pegar um táxi de volta pro carro e então voltar para casa.

Enfim, Niels Bohr disse: "Qualquer predição é arriscada, principalmente sobre o futuro.", ok mas não precisava exagerar!

The Role of Programming

2007-05-13T14:41:00.000-07:00

Não deixe que a brevidade desta exposição feita por Gerald Jay Sussman mascare a enorme quantidade de conhecimento embutida.

Perl 6, onde está você?

2007-03-21T18:55:00.000-07:00

Pode ser uma guerra de egos, talvez tão intensa quanto as guerras por motivos religiosos, times de futebol e partidos políticos. Mas querendo ou não, Perl é uma linguagem que tem seu espaço. Desenvolvo em Perl desde 1999 e posso listar algumas dos maiores mal entendidos sobre essa liguangem.

Perl é uma linguagem voltada a CGI

Esse é um engano tanto quanto interessante, talvez porque a alguns anos muito do dinamismo em websites era fornecido por um script em Perl rodando como CGI num servidor Apache. É interessante porque a Perl não foi concebida como uma linguagem voltada a geração de páginas da web e sim como uma linguagem para processar e extrair relatórios, seu nome já diz isso PERL - Pratical Extraction and Report Language. O que eu posso concordar é que lidar com CGI, também é lidar com processamento de texto, principalmente inputs de usuários e etc, mas ao contrário de outras linguagens como PHP, a Perl não foi criada com esse objetivo em mente, pelo menos não explicitamente.

Perl não é uma linguagem para sistemas robustos, grandes, comerciais e etc.

Esse argumento é bastante complicado de se refutar. Primeiramente o que faz uma linguagem adequada para tais tarefas? Minha experiência com sistemas comerciais é pequena, mas há projetos em Perl muito sérios e robustos, um exemplo é o Ensembl um visualisador de genomas, implementado sobre a famosa BioPerl. Para saber mais vale o artigo do admirável Lincoln Stein - How Perl saved human genome. Por fim, as opiniões podem variar, mas creio que o Projeto Genoma Humano é um projeto sério e grande e merece respeito. Até hoje Perl é extensamente utilizada na pesquisa em Bioinformática, e para uma referência final talvez uma olhada nas Perl success histories.

Perl é uma linguagem estranha e sua sintaxe é ilegível.

A mim isso não é argumento, LISP tem uma sintaxe talvez mais estranha e é uma linguagem levada a sério. E mais de uma vez quando deparo com outras linguagens devo tirar um tempo para me adaptar à sintaxe da linguagem. Na verdade até Java pode ser escrita de maneira confusa e ilegível, e pela minha experiência não é a sintaxe da linguagem, ou seja, as instruções que a tornam ilegível e sim o programador. Utilizar recursos obscuros da linguagem, mesmo quando não são necessários, é uma contradição a mais de uma das recomendações do livro Elements of Programming Style, veja este (PDF) para um pequeno resumo. E finalmente a melhor maneira de refutar este argumento é citar o The International Obfuscated C Code Contest, oras apesar deste concurso podemos citar a linguagem C utilizada em grandes projetos, portanto não é a legibilidade de um programa que define a qualidade da linguagem em que está escrito, ao menos para casos normais, para outros casos que tal Malbolge? E Perl também teve o seu Obfuscated Perl Contest para os curiosos.

Perl falha no paradigma Orientado Objeto.

Este argumento já é pronunciado por pessoas com um pouco mais deconhecimento em Perl e realmente pega num ponto "fraco" da Perl. Coloco a palavra fraco entre aspas porque Perl realmente não é uma linguagem concebida como orientada a objetos (OO de agora em diante), mas isso aparentemente não é um problema, e porque não? Bom, ao ler o livro Object Oriented Perl de Damian Conway você se surpreenderá de como Perl pode ser utilizada dentro do paradigma OO com muita flexibilidade e integridade. Projetos como a BioPerl (citada acima) são todos implementados utilizando a Perl em OO, quase todos os 11348 (em 21/03/2007) módulos hospedados no CPAN possuem uma interface OO. Sim, Perl não é nativamente OO, mas isso não parece ser algo tão grave assim, há vários módulos que tornam a Perl mais OO. No entanto vale o lembrete que OO é um paradigma e não uma linguagem específica, ou implementação de uma linguagem, é uma maneira de se trabalhar e pensar! Visto que a Perl nos dá maneiras de codificar de acordo com este paradigma, então basta disciplina, veja a entrevista de Donald Knuth onde ele cita exatamente isso. Mas porque qual é a razão deste artigo? O título dele faz referência à Perl6, e quem é a Perl6? A Perl6 é uma grande promessa, é a reescrita da Perl (formalmente conhecida como Perl5) pela comunidade de desenvolvedores ("Perl 5 was my rewrite of perl... I think this should be the community's rewrite of perl, and the community's rewrite of the community."), note a data do artigo 19/07/2000. Dentro das novas especificações da Perl6 há:

Tipagem forte
Metadados em variáveis, valores, subrotinas...
Expressões regulares em Streams
Argumentos nomeados

E muito mais, para aqueles que não sofrem do coração os Apocalypse Docs podem ser um lugar bom para se perder. E mais, a Perl6 irá fazer uso do Parrot, uma Virtual Machine capaz de executar o código da Perl6. Ops?! Virtual Machine para Perl? Onde mais ouvimos falar de Virtual Machine? Sim, Java! Java faz uso de uma Virtual Machine e logo a Perl também o fará. Enfim, minha ansiedade pelo release 1 da Perl6 é muito grande. Dadas tantas novidades e tanta potencialidade eu sinto que a Perl6 pode muito bem ocupar um lugar de distinção no meio comercial. Pois neste caso, ela estará de acordo com algumas exigências como tipagem, bytecode, virtual machine e etc, que são utilizadas para refutar o uso da Perl nestes ambientes. Apesar de ainda não existir um release oficial da Perl6 há uma implementação teste, projeto iniciado por Audrey Tang, chamada Pugs, que não é nada mais que a Perl6 implementada em Haskell. Eu pessoalmente não tive opurtunidade de utilizar o Pugs da maneira que gostaria, mas recomendo a todos um teste drive, afinal de contas não há mal em experimentar novas linguagens, já foi dito que um programador pragmático aprende uma nova linguagem por ano. Não deveria ser novidade que depois de tanta promessa eu exclame "Perl6, onde está você?" Keep Coding Marco.

Modelando o mundo

2007-03-12T21:55:00.000-07:00

Fica aqui o início deste blog. Nada mais que YAB (Yet Another Blog) sobre Programação. Creio que os posts mostrarão os objetivos e os leitores definirão os rumos. Keep Coding. []' s Marco