Sunday, December 19, 2010

Configure Tomcat to accept HTTP PUT command

Tomcat by default is not enabled for HTTP PUT command. But, it can be easily configure to support it.

First, in order to allow Tomcat to accept commands other than HTTP GET and PUT we should add to Tomcat’s web.xml the following init param:

<servlet>
<servlet-name>default</servlet-name>
<servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class>
<init-param>
<param-name>debug</param-name>
<param-value>0</param-value>
</init-param>
<init-param>
<param-name>listings</param-name>
<param-value>false</param-value>
</init-param>
<init-param>
<param-name>readonly</param-name>
<param-value>false</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>

Note, that Tomcat’s web.xml is usually located under TOMCAT_HOME/conf/web.xml

We would probably want to allow only specific users to access the PUT command, since it is a command that allows putting resources on our server. Therefore, we will add a user to the file: tomcat-users.xml:

<?xml version='1.0' encoding='cp1252'?>
<tomcat-users>
<user name="admin" password="admin" roles="admin" />
</tomcat-users>

Note, that Tomcat’s tomcat-users.xml is usually located under TOMCAT_HOME/conf/tomcat-users.xml

Finally, we should add a security constraint to the web.xml of the specific web application, for which we want to allow PUT command:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">
<security-constraint>
<web-resource-collection>
<web-resource-name>Demo App</web-resource-name>
<url-pattern>/*</url-pattern>
<http-method>PUT</http-method>
</web-resource-collection>
<auth-constraint>
<role-name>admin</role-name>
</auth-constraint>
</security-constraint>
<login-config>
<auth-method>BASIC</auth-method>
<realm-name>Demo App</realm-name>
</login-config>
<security-role>
<description>Role for restricted resources</description>
<role-name>admin</role-name>
</security-role>
</web-app>

Note for the following things:

  • The web.xml of your web application is usually located under: TOMCAT_HOME/webapps/YOUR_WEBAPP/WEB-INF/web.xml
  • In order to use the PUT command, you should pass username and password as part of the request (admin/admin in our example).
  • Note that we enable PUT command for the whole web application, but it is also possible to allow the PUT command only to a part of our web application, by using the “url-pattern” option.

Saturday, December 18, 2010

Mobile Operator/Carrier detection using MaxMind – Second Part – Creating carriers JSON file

In this post we saw how the first steps to mobile operator/carrier detection. We saw how the data of carriers and carriers mapping is stored in database.
Now we will see how to convert this data to a JSON file. Our mobile operator/carrier detection module will be probably be used by an Ad server. Ad servers are usually not connected to database (you need Ad server to be robust and work under high pressure). Therefore, we would like to be able to load the carriers information from a simple file.
For this purpose, we will create a simple class named CarrierDataHolder that will host all the mobile operator/carrier mapping in a single map. The map is constructed from 2 inner classes: a composite key named Key and value named Value.
These are the members of the map Key:
  • countryCode
  • isp
  • org
These are the members of the map Value:
  • carrierId
  • name
Note, that for the Key of the map, we implement “hashCode” and “equals” methods, otherwise the map would not function properly.
By using this map, we later be able to get any carrier simple by supplying country code, isp and organization. The country code isp and organization will be extracted from the user ip using MaxMind.
The CarrierDataHolder contains 2 important methods:
  • loadCarriersFromDB – This method gets a database connection and loads all the carrier and carrier mapping information to the carrierMapping map.
  • toJSONFile – This method stores all the data loaded to carrierMapping map to a JSON file. The data is saved to JSON using XStream open source project.
Let’s have a look at the CarrierDataHolder class:
package com.bashan.blog.geo;
import com.thoughtworks.xstream.XStream;
import com.thoughtworks.xstream.io.json.JettisonMappedXmlDriver;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.sql.*;
import java.util.HashMap;
import java.util.Map;
/**
* @author Bashan
*/
public class CarrierDataHolder {
 
public final Map<Key, Value> carrierMapping = new HashMap<Key, Value>();
 
public void loadCarriersFromDB(Connection conn) throws SQLException {
carrierMapping.clear();
Statement stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery("select c.carrier_id, c.country_code, c.name, cm.isp, cm.org from carrier c inner join " +
"carrier_mapping cm on c.carrier_id = cm.carrier_id order by c.carrier_id");
  while (rs.next()) {
    int carrierId = rs.getInt("carrier_id");
String countryCode = rs.getString("country_code");
String name = rs.getString("name");
String isp = rs.getString("isp");
String org = rs.getString("org");
carrierMapping.put(new Key(countryCode, isp, org), new Value(carrierId, name));
}
}
 
private XStream getXStream() {
XStream xstream = new XStream(new JettisonMappedXmlDriver());
xstream.setMode(XStream.NO_REFERENCES);
xstream.alias("carrierMapping", Map.class);
xstream.alias("key", Key.class);
xstream.alias("value", Value.class);
  return xstream;
 
public String toJSON() {
XStream xStream = getXStream();
  return xStream.toXML(carrierMapping);
 
public void toJSONFile(File file) throws IOException {
XStream xStream = getXStream();
FileWriter fileWriter = null;
  try {
fileWriter = new FileWriter(file);
xStream.toXML(carrierMapping, fileWriter);
} finally {
      if (fileWriter != null) {
fileWriter.close();
}
}
}
 
public static class Key {
 
private String countryCode;
private String isp;
private String org;
 
public Key() {
}
 
public Key(String countryCode, String isp, String org) {
  this.countryCode = countryCode;
  this.isp = isp;
  this.org = org;
}
 
public String getCountryCode() {
  return countryCode;
 
public void setCountryCode(String countryCode) {
  this.countryCode = countryCode;
 
public String getIsp() {
  return isp;
 
public void setIsp(String isp) {
  this.isp = isp;
 
public String getOrg() {
  return org;
 
public void setOrg(String org) {
  this.org = org;
}
 
@Override
public boolean equals(Object o) {
  if (this == o) return true;
  if (o == null || getClass() != o.getClass()) return false;
Key key = (Key) o;
  if (countryCode != null ? !countryCode.equals(key.countryCode) : key.countryCode != null) return false;
  if (isp != null ? !isp.equals(key.isp) : key.isp != null) return false;
  if (org != null ? !org.equals(key.org) : key.org != null) return false;
  return true;
 
@Override
public int hashCode() {
  int result = countryCode != null ? countryCode.hashCode() : 0;
result = 31 * result + (isp != null ? isp.hashCode() : 0);
result = 31 * result + (org != null ? org.hashCode() : 0);
  return result;
}
 
public static class Value {
 
private int carrierId;
private String name;
 
public Value() {
 
public Value(int carrierId, String name) {
  this.carrierId = carrierId;
  this.name = name;
 
public int getCarrierId() {
  return carrierId;
 
public void setCarrierId(int carrierId) {
  this.carrierId = carrierId;
 
public String getName() {
  return name;
 
public void setName(String name) {
  this.name = name;
}
}
}


The data loaded by the loadCarriersFromDB method is taken from carrier and carrier_mapping tables. This post explains how they are created.

Let’s have a look at a small test program that loads all carriers information and stores it to a JSON file. Note that we assume the name of the database is adserver:
public static void main(String[] args) {
CarrierDataHolder carrierDataHolder = new CarrierDataHolder();
Connection conn = null;
  try {
String userName = "root";
String password = "root";
String url = "jdbc:mysql://localhost:3306/adserver?useUnicode=true&characterEncoding=UTF-8";
Class.forName("com.mysql.jdbc.Driver").newInstance();
conn = DriverManager.getConnection(url, userName, password);
carrierDataHolder.loadCarriersFromDB(conn);
carrierDataHolder.toJSONFile(new File("c:\\data\\carriers.json"));
} catch (Exception e) {
e.printStackTrace();
} finally {
  if (conn != null) {
    try {
conn.close();
} catch (SQLException sql) {
sql.printStackTrace();
}
}
}
}

You can download the class CarrierDataHolder by pressing this link. Note that this class is using XStream open source project. Therefore you will have to download XStream jar files as well (xstream and jettison).
Here is the mobile operators/carriers detection JSON file generated by using the CarrierDataHolder class.

Thursday, December 16, 2010

Show IPv4 instead of IPv6 on DOS ping command

If you are seeing something like this when doing PING on your DOS command like:


It means that your ping command is showing you the IPv6.
In order to see IPv4 you should use the “-4” parameter.
For example: “ping www.google.com –4”
This is how it looks:

Monday, December 6, 2010

Panda Antivirus – Cloud Antivirus

If you are looking for a free, simple and easy to use antivirus you can check Panda Cloud Antivirus. It has all the basic functionalities of a free antivirus and it seems to work pretty well in terms of performance. The interesting thing about it, is that it claims to be: Cloud Antivirus. Meaning, that it collects all ever-expanding knowledge from all the users connected to it. If one of the users on the cloud makes a contact with a virus, the rest of the community is being blocked from this virus in a matter of 6 minutes, unlike other free antiviruses, which usually update their database only on a daily basis. Users are actually updated with all the latest viruses all the time.
Panda Cloud Antivirus is free, but if you want a more comprehensive protection you can download Panda Cloud Antivirus Pro.
I used Panda Cloud Antivirus free version for about an year and I can tell that it is a light weight antivirus that doesn’t consume too much computer resources. It is a very easy to operate and doesn’t give you endless warnings and questions. It simply runs on the background, and does its job. The only reason I switched from Panda Cloud Antivirus to Kaspersky is because I had a chance of getting a free serial.
Here is how the main screen of Panda Cloud Antivirus looks like:

panda_cloud_av_1.3_main

As you can see, it is very neat and simple.

The differences between Online Advertising and Mobile Advertising

Online advertising has become commodity long time ago. With the rapid growth of mobile devices market, mobile advertising is becoming more and more popular, and starting to get a significant share of advertising budgets.

We advertise on the internet and we advertise on mobile devices. Both are not really that different. Mobile devices are becoming smarter and the experience is getting quite similar to online browsing. In both online and mobile worlds, publishers show text or image ads on their sites/apps and count analytics.

So, what is really the difference between the online and the mobile?
The truth is, on first sight, there is no much difference. From one hand we have advertisers that want to reach targeted audience, from the other hand we have publishers that want to monetize their business.

When drilling down a little deeper to technical levels we are starting to see, that although there is a big resemblance between both worlds, there are are also several significant differences:
  • Mobile carrier detection - Many advertisers are not only interested in advertising on specific countries, but also on specific mobile carriers (for example, AT&T, Sprint, Orange, etc.). Therefore, mobile ad network should have the ability not only to know from which country you are coming, but also from which mobile carrier you are coming. Of course, if you are NOT coming from any mobile carrier (by using WAP), you are probably using WiFi on your mobile device. So, knowing to distinguish between WAP and WiFi can also give additional way of targeting your mobile audience.
  • Brand and Device detection - On the online world, you have several popular browsers like: Firefox, Explorer, Google Chrome, Opera, etc. The mobile world is much more varied. There are many brands like: Nokia, Apple, Samsung, Motorola and many more and for each brand there are many devices growing on a daily basis. Advertisers would like to target their campaigns according to brand and/or device. Therefore, mobile advertising platforms must detect from which brand and device each request is coming.
  • Location based targeting – On the online advertising world, at first, campaigns used to be targeted to specific countries. Then systems evolved, and allowed the possibility to target campaigns according to a specific state or even specific city. On the mobile world, this is just not enough. Modern mobile devices allows knowing exact user location. This gives the possibility of creating very accurate geographical campaigns, up to a region of few kilometers and even less.
  • Screen size detection – On the online world, most users have high resolution screens. You create a single size banner that fits all. One the mobile world, single size banner is not enough. There are so many devices with so many screen sizes. For each campaign you have to create 4 different sizes of banners. You have to detect the exact size of each device and be able to fit each device the most appropriate banner.
  • Conversion analysis – One of the evolutions of online advertising was conversion analysis. Advertisers want to keep track after the converting users and know information about them. This helps them refine the targeted audience and eventually get higher ROI. But, advertisers are not so experienced with technology. Therefore, advertising platforms help advertisers to keep track of their converting users, by doing simple integration process, usually done with some client side JavaScript code. The process is very simple: You get a JavaScript snippet. You put it in some place(s) on your page(s) and you are ready to go. By taking advantage of JavaScript and cookies, the hard work is being done for you behind the scenes. On the mobile world things are quite different. Usually, you cannot reply either on JavaScript nor cookies, since not all devices support them. Therefore, the conversion tracking process becomes a harder task to accomplish (Usually, integration process involves some pixel tracking or request needed to be sent from advertiser to the mobile ad network). But, after integration with advertiser is done, more interesting information can be collected regarding converting users. You can tell if users come from a specific mobile carrier are converting more than another carrier, or if users from a specific brand or even device are generating a higher conversion.

Clear Explorer search history in Windows 7

Windows 7 stores latest search queries history. You can see these search history by clicking on the search box. Here is an example of how it looks:


The search history can be easily cleaned by using Windows Registry Editor:
  • Load Windows Registry Editor by writing “regedit” on the “search programs and files” edit box (press the “Start” button to see it). Here is an example of how it looks:

  • When Windows Registry Editor is loaded go to the following registry key:
HKEY_CURRENT_USER/Software/Microsoft/Windows/CurrentVersion/Explorer/WordWheelQuery
Right click on “WordWheelQuery” and choose “delete” option from the popup menu. Here is how it look on Windows Registry Editor:
  • That’s it. Click on your search box and make sure that its history data is empty.

Wednesday, December 1, 2010

Mobile Operator/Carrier detection using MaxMind – First Part – Database information

One important key targeting of mobile advertising networks, is the ability to target mobile campaigns for specific mobile operators/carrier. In order to it, we have to be able to detect the mobile operator of a given IP. I have been looking for quite some time (the passed 2 years) for a decent solution for mobile operator detection. The only decent solution (there are not much services like that) came from Quova. The problem with Quova, is that they don’t give you an API and some binary file (like most other geo/device targeting services). They insist you install dedicated server solution, that you have to query by HTTP, in order to get the information you want. If you have several machines, you have to start install Quova solution on each and every one of them.
Since we didn’t like Quova solution on the company that I work for (Mobile Ad Network), and we couldn’t find any other good solution, we decided on developing our own in-house solution. The solution is based on user IP and ISP and Organization data as it extracted from MaxMind geo service.
This is in general the detection processes:
  • Get client IP.
  • Use MaxMind to extract ISP and Organization from client IP.
  • Check what is the corresponding mobile operator/carrier for the given combination of ISP and Organization.
The solution is quite simple and neat. The only problem with it, is that it is needed to be maintained on a regular basis, since Geo information is being changed all the time.
In order to be able to identify mobile operators, we first need to have two tables containing the following information:
  • Mobile operators/carriers table, containing all mobile operators in the world. You can have a look at this post in order to see how I built the information in this table.
  • Mobile operators and their ISP and organization. This table maps between mobile operators and their corresponding ISPs and organizations. Building this table is a hard work involving data mining, research and continuing refinement and update process. During the time of this post, the data in this table is quite good, but not full (not all countries/mobile operators are mapped).
Note for:
  • The data in these tables is loaded to memory, since we need good performance.
  • MaxMind updates it’s ISP and Organization data on a monthly basis.
  • ISP and Organization is being changed all the time. The changes are not drastic, but they happen all the time, so data is needed to be maintained, and there is a need to keep refining the mapping between mobile operators/carriers and combinations of ISP and Organizations.
Let’s have a look at the structure of the mobile operators/carriers table. The table is called “carrier” and contains 3 columns:
  • carrier_id – The id of the mobile operator/carrier. This is not some universal id. It is an arbitrary id used only by the system.
  • country_code – The country code of the mobile operator/carrier.
  • name – The name of the mobile operator/carrier.
The SQL script that creates this table:
CREATE TABLE `carrier` (
`carrier_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`country_code` char(2) NOT NULL,
`name` varchar(100) NOT NULL,
PRIMARY KEY (`carrier_id`)
) ENGINE=InnoDB;
This script contains both the scrip that creates “carrier” table as well as its data. It contains all the mobile operators in the world. This data has to be maintained (carriers are being added or merged from time to time), but not very often.

This is the structure of the table that maps ISPs and Organizations to mobile operators/carriers. The table is called: “carrier_mapping” and also contains 3 columns:
  • carrier_id – The id of the mobile operator/carrier, for which we would to like to map a combination of ISP and Organization.
  • isp – ISP corresponds to carried_id.
  • org – Organization corresponds to carrier_id.
The SQL script that creates this table:
CREATE TABLE `carrier_mapping` (
`carrier_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`isp` varchar(50) NOT NULL,
`org` varchar(50) NOT NULL,
PRIMARY KEY (`carrier_id`,`isp`,`org`),
CONSTRAINT `FK_carrier_mapping_carrier_id` FOREIGN KEY (`carrier_id`) REFERENCES `carrier` (`carrier_id`)
) ENGINE=InnoDB;
This script contains both the scrip that creates “carrier_mapping” table as well as its data. It contains a decent amount of mapping of most large mobile operators/carriers in the world. Note that the data here may not be so accurate in a few months or so.