Showing posts with label Database. Show all posts
Showing posts with label Database. Show all posts

Sunday, February 17, 2019

Migrating from DynamoDB to Postgres

The mass popularity of NoSQL databases has also resulted in it being used in all use cases without understanding the suitability for the use case. One fundamental rule that is usually forgotten is that the NoSQL databases are designed around queries. Initially, the schema is evolved based on the need for initial business use cases. However, the business use cases evolve and change in multiple ways and soon the new needs of interacting with the database become unwieldy. This is the fundamental problem that people usually hit wiht NoSQL databases.

Where DynamoDB gets into trouble:
  • As the business use cases evolve and change the need to query the database in multiple ways arise. DynamoDB is not easy to query if it is not queried based on the partition key. One can build indexes but there is a cost associated with it. Filters can be used to query the data but that involves scan and as the data grows it starts becoming costly both in terms of money and time.
  • DynamoDB is schemaless so with time the data evolve in multiple ways. In older data records, it is quite possible that the fields might be missing or might have a different interpretation. Developers keep handling them in the deep layers of code to keep the world moving. However, soon it results in too many if-else statements. Migration is a pain to handle such cases, however, one has to be ready for missing fields and handle them with suitable defaults.
  • There is no relationship integrity so it's easy to put wrong data in relationships and it's very difficult to figure out even if something like that has happened. In SQL also it's possible to put a wrong key with a valid foreign relationship but still in terms of integrity SQL provides much better primitives.
  • This will be a repetition of the above points but a different perspective. As it's fine to add any kind of data in the table, people start putting all kind of data in it. Imagine that everyone in the world is given all kind of freedom. Sounds romantic. However, soon ti will be chaos as everyone is living in all different ways. 
Sample code for migration

import boto3
import psycopg2

# Create a connection to DynamoDB. Please fill the required keys. A better way to do is 
# to put it in config file and pass it through. In AWS environment it's better to use Roles
dynamo = boto3.client('dynamodb',aws_access_key_id='<access-key>', \
                      aws_secret_access_key='<access-secrte>', \
                      region_name='<region>')

# Create the database connection
db = psycopg2.connect(host="<db_host>,database="<db>", user="<user>", password="<pass>")
dbCurr = db.cursor()

#Assume we have a user table in dynamoDB and insert it into the postgres Users table
user = dynamo.query(TableName="User", KeyConditionExpression ="Email = :email", \
                               ExpressionAttributeValues = {":email": { 'S': email }})  

email = user.Items[0]['Email']

#Note the returning id so that we can use the id of the newly persisted record.
#This can be used to create foreign key relationships for further table
userSql = 'INSERT INTO users(email) VALUES(%s) returning id'
userValues=(email)

dbCurr.execute(userSql,userValues)
dbUserId = int(dbCurr.fetchone()[0]) 

db.commit()
dbCurr.close()

Some more stories from the web:

Tuesday, September 11, 2018

SQL vs NoSQL

Image Source: Pixabay
SQL and NoSQL are two important choices for application data storage needs. There is a lot of confusion about which one to choose and what is a good fit. There are buzzwords on both sides which make the choices more confusing. How to choose one over another?

Tuesday, February 27, 2018

Hibernate Envers

Please follow Rest API Server to see the big picture and GitHub repo details.

Hibernate Envers helps in creating an audit trail of the records in the database. Hibernate Envers audits all the attribute changes in an audit table. To enable the support for Hibernate Envers, add the dependency

    compile('org.hibernate:hibernate-envers')

Add the appropriate version. If you are using Spring boot dependency management then the version will be automatically taken care by Spring boot.

Sunday, February 25, 2018

Liquibase Tutorial

Photo credit: Pixabay.com


Please follow Rest API Server to see the big picture and GitHub repo details.

Liquibase helps in database schema management. The schema can be defined in XML, YAML, JSON and SQL format. It supports multiple databases. Also, the evolution of the database can be managed through liquibase using the concept of changesets.

In build.gradle add

  compile('org.liquibase:liquibase-core')

In application.properties add

Monday, June 6, 2016

Streaming data from Result Set

This example shows a way to stream the database records in a JSON format. This post is done against postgres db and a table is assumed. However you can point to any table structure and db after doing required adjustments.


public class StreamingService {

  public void handleRequest(String sql, OutputStream op) throws IOException 
{
   //Initialize the driver
   try {
 Class.forName("org.postgresql.Driver");
   } catch (Exception e) {

   }

Tuesday, April 19, 2016

Mysql Cheat Sheet

To List all the databases, their tables and the number of records in each table


select table_schema,table_name,table_rows from information_schema.tables;

displays in following format

+---------------------------+----------------------------------------------------+------------------+
 | table_schema               | table_name                                                   | table_rows       |
+---------------------------+----------------------------------------------------+------------------+

Note: The table_rows here are approximated rows and may not show the correct number of rows. Be careful.

Extended Display

Select * from abc \G;

\G results in extended display

Dumping database

mysqldump -u<username> -p<password> <db_name>  >   dump.sql

dump.sql will contain the dump of db

Wednesday, April 6, 2016

Comparing Mysql databases

This is often a need to compare two databases in terms of schema and data. This might be required to compare with production databases or in test environment to ensure that the data values has not changed. For the same, Mysql has a utility called mysqldbcompare. Let's see how to use it. First the installation.

Installation

You can either download from you os repositories or can download it directly from the mysql site. Do a search for mysql-utitlies and you can reach to the page. One way to do it in Linux is as follows

wget https://dev.mysql.com/downloads/file/?id=458907wget https://dev.mysql.com/get/Downloads/MySQLGUITools/mysql-utilities-1.5.6.tar.gz

This will download the file in your current directory. Than unzip it and run the python command to install it.

tar -xvf mysql-utilities-1.5.6.tar.gz
cd mysql-utilities-1.5.6
sudo python setup.py install

Now let's say we want to compare two database db1 and db2 in a machine called s1. The command goes like

mysqldbcompare --server1=<user>:<password>@s1:3306 db1:db2

Here <user> is the db user and <pass> is the password of the user. If the db are residing in two different machine called s1 and s2 the command becomes

mysqldbcompare --server1=<user>:<password>@s1:3306 --server2=<user>:<password>@s2:3306 db1:db2

By default it stops at first failure. If you want to run it complete than provide --run-all-tests option

mysqldbcompare --server1=<user>:<password>@s1:3306 --server2=<user>:<password>@s2:3306 db1:db2  --run-all-tests

It comes with a lot of switches. For details refer to the mysql documentation page. 

The downside of this approach seems to be that the comparison cannot be configured to handle certain columns of the table only. This becomes important as sometimes one wants to ignore audit columns like create and update dates. 

Friday, November 7, 2014

DELETE vs TRUNCATE vs DROP

Delete is used to delete the records from a table. A where clause can be put in delete statement. For example if you want to delete all records from a users table, than the SQL will be

delete from users;

A where clause can be put to delete only users having first_name start with a

delete from users where first_name like 'a%';

With delete call, triggers are fired and indexes are updated. Also it puts a lock on the table. If the table contains a large number of records, delete statement can take a good amount of time. In those

Thursday, October 16, 2014

Exceptions while Batch processing in Postgres

If you are encountering the following exception

Caused by: org.postgresql.util.PSQLException: ERROR: current transaction is aborted, commands ignored until end of transaction block

you have been hit with the way postgres handles a batch. When postgres is handling a batch of queries,  even if one of the query while executing fails with an exception, Postgres aborts the whole

Sunday, October 12, 2014

In Memory Databases

In memory databases are different from traditional databases in the sense that in memory database sits in memory and traditional databases stores the data in files system. As the data sits in in-memory, in memory databases are much faster than traditional databases. The primary reason for performance is that in memory databases do not have to seek the disk. 

In terms of ACID properties, in -memory databases satisfy all the requirements apart from durability. As the database exist in memory, in case of power failure, the database can loose the data. However their are many techniques which are employed to handle the situation. The involves from maintaining

Saturday, October 11, 2014

Multicolumn Indexes in Postgres

Multicolumn index means putting an index on a combination of two column. Let's look into an example to see how multicolumn index works. Let's say we have user table with following columns

human
  • id - primary key of record
  • day_of_year - Day from 1 to 366, when the human was born
  • first_name - First name given to human

Let's say we want to build this table for all the humans on earth.  Quite a number of records. Before we look into multicolumn index, let me be very clear that the indexing should be driven by the kind

Query Plan in Postgres

When a query is issued to a database, the query is executed as per query plan. To fetch the relevant results, database has to scan the records and filter them based on criteria. For example if for fetching the results of query, the database can fetch the records from indexes or will have to do a complete search on table. 

Understanding query plans is an important tool to understand and diagnose the query performances. Most of the database provide the tools to study query plans including Oracle. In this post we will look into how to study query plans of Postgres but conceptually it's similar in other databases.

Let's take and example of a table and understand how we can study the query plan. Let's say we have

Tuesday, October 7, 2014

Logging Queries in Postgres

Postgres logs can be enabled by turning switches in postgresql.conf file. The location of

postgresql.conf file is at

Windows : <Installation Directory of Postgres>/PostgreSQL/<version>/data/postgresql.conf
Linux : /etc/postgresql/<version>/main/postgresql.conf

Look for the following line

#log_statement = 'none'         # none, ddl, mod, all

Saturday, August 16, 2014

Java Database Connectivity (JDBC)

JDBC stands for Java Database connectivity. It's an API which helps Java based applications to interact with RDBMS using SQL and PL/SQL. JDBC is part of specification and defined the interfaces. The implementation is provided by the jdbc vendors. For example you will use a different JDBC driver for connecting to Oracle and DB2. However if you are writing your code against the JDBC base interfaces, the program is guaranteed to run against both the databased by just switching the implementation.

Let's look into a typical JDBC code

Student student = new Student();
student.setName("Amitabh");
        
try{
   //Load the database driver class
   Class.forName("org.hsqldb.jdbcDriver");
   }catch(ClassNotFoundException cfe){
       System.out.println("Driver not found");
       System.exit(0);
  }
        
Connection conn = null;
PreparedStatement stmt = null;
try{
   //Get the connection
   conn = DriverManager.getConnection ("jdbc:hsqldb:hsql://localhost","sa","");
   
   //Create a statement
   stmt = conn.prepareStatement ("insert into STUDENT (name) values (?) ");
   stmt.setString(1, student.getName());

   //Execute statement
   stmt.execute();
   }catch(SQLException se){
        System.out.println("Problem with data insert");
   } finally{
       //Make sure everything is closed
       try{
           if(stmt != null) 
             {
                stmt.close();
              }
           if(conn != null)
             {
                conn.close();
             }
       }catch(SQLException se) {
       }    

In a nutshell, the following steps are done
  • Define connection parameters
  • Open the connection
  • Specify the statement
  • Prepare and execute the statement
  • Process any exception
  • Close the connection
Many modern day applications use Spring with Hibernate to deal with database connectivity as they provide easy and maintainable way.

Friday, August 15, 2014

Hypersonic

Hypersonic SQL is a very good database for development purpose. Please check at http://hsqldb.org/ To use hsqldb just drop in the jar in your library path and start it as a java application from your IDE. It comes with a data browser also which can help in looking into the database. To start the database in eclipse see the following video

The details of connecting the database are:


driver_class=org.hsqldb.jdbcDriver
url=jdbc:hsqldb:hsql://localhost
username=sa
password=""

H2 Database

H2 Database is another open open source database with a very small foot print. The database can be downloaded from http://www.h2database.com

Download H2 database. Windows version comes with an installer for H2 database otherwise you can download the zip version and unzip it at a location. Go to the bin directory of unzipped location and issue the command based in platform.

In Linux in a terminal issue 

./h2.sh. 

It will open a browser. In the browser select Generic H2 (Server) in saved settings menu. Accept the defaults. However you are free to change the defaults also. Click on connect. This will take you to a screen where you can do your database related stuff.

Database connection parameters for H2 Database is:

driver_class=org.h2.Driver
url=jdbc:h2:tcp://localhost/~/test
username=sa
password=""

SQL - How to update one field of existing date

 Let's say we want to set all salary dates to 1st of the month from an environment where it was staggered for different people at different days of the month.
To update an existing date in such a way, that only one of the field of the date is changed and rest all remain constant (Tested in Mysql).
update employee set salary_date = 
STR_TO_DATE(concat(month(salary_date),'-',01,'-',YEAR(salary_date)), '%m-%d-%Y') ;
Note that only the month field is replaced here.

Host errors in MySQL

Sometimes you get the following error in mySQL

There is an error in get connection. <Host> is blocked because of many connections error; unblock 
with 'mysqladmin flush-hosts'

The happens because of certain number of connections failing from the host, mysql assumes that someone is trying to break in and it blocks the host from connecting further. If the host has to be allowed than on the database server run

mysqladmin flush-hosts

Sometimes running mysqladmin gives the following error:

mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'ODBC'@'localhost' (using password: NO)'

In that case, run with

mysqladmin -h 127.0.0.1 -u root -p flush-hosts

and be ready with the root password

You can bump up the max_connect_errors also which by default is 10. It means after 10 bad connections, the server will refuse to honour the connection request of the particular host. To bump up the connection, do following in mysql window using admin credential

set GLOBAL max_connect_errors = 100

Access Problem

When following error message occurs,

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
than, start mysql with the following arguments

mysql -h 127.0.0.1 -u root -p

Where root is the user with which you want to login.

Wednesday, August 13, 2014

Spring Data Access

Any non trivial enterprise application usually have a persistence mechanism and usually it’s a SQL compliant database. Please note that Spring does not provide any native support for database access. Spring does not provides any SQL like language to connect to database. What Spring provides is integration to various data access framework which includes
  • JDBC
  • Hibernate
  • iBatis
  • Apache Object Relationship Bridge
  • JPA
  • Toplink

Spring provides the following features to handle the database:
  • Spring provides easy way to handle database interaction using one of the data access mechanisms. It will handle the boiler plate code so that application developer can concentrate on application logic.
  • Managing connection is a non trivial process which Spring handles in an easy way.
  • Transaction boundaries can be handle in an easy way using annotations.
  • Spring promotes unchecked exceptions philosophy. Remember in your JDBC interaction code how many times you have done anything useful with the SQLException. Pretty much nothing apart putting a message in console. Spring catches the SQL exception and converts into unchecked exceptions.  Another thing Spring does is to check the vendor specific errors in exceptions and converts it into a uniform exception hierarchy.

Let's create a Student class and persist it.

Student.java

public class Student {    
  protected String name;

  //Getters and setters

Let's use a database. I am using Hypersonic, but you can use can any database

SQL for creating Student

CREATE TABLE STUDENT ( NAME VARCHAR )

Let's first look into the classic way of persisting the data.

Student student = new Student();
student.setName("Amitabh");
        
try{
   Class.forName("org.hsqldb.jdbcDriver");
   }catch(ClassNotFoundException cfe){
       System.out.println("Driver not found");
       System.exit(0);
  }
        
Connection conn = null;
PreparedStatement stmt = null;
try{
   conn = DriverManager.getConnection ("jdbc:hsqldb:hsql://localhost","sa","");
   stmt = conn.prepareStatement ("insert into STUDENT (name) values (?) ");
   stmt.setString(1, student.getName());
   stmt.execute();
   }catch(SQLException se){
        System.out.println("Problem with data insert");
   } finally{
       try{
           if(stmt != null) {stmt.close();}
       if(conn != null) {conn.close();}
       }catch(SQLException se) {}    

Look how much verbose is the code. The program in statement is:

  • Define connection parameters
  • Open the connection
  • Specify the statement
  • Prepare and execute the statement
  • Process any exception
  • Handle transactions
  • Close the connection

Note that apart from the step of specifying the statement, rest all is boiler plate code which is repeated endlessly.Now let's see how Spring handles this situation. Before we move further let's look into Spring DAO philosophy. Spring promotes a pattern called Data Access Object(DAO) pattern, where Spring promotes interface based approach. This leads to flexibility in the programming model as it facilitates change of concrete implementation with ease. Let's write the interface
StudentDao

public interface StudentDao {

   public void saveStudent(Student student);    
}

And if we are dealing with JDBC based interaction to database, the implementation class will look like

public class StudentJdbcDao implements StudentDao{

   public void saveStudent(Student student) {
   //Here we have to provide the implementation.
   //We can pull our JDBC interaction code here
  //but we do not achieve much with this apart from an indirection because of interface.
   }
}

Let's take a step back and understand how Spring handles this kind of situation. Spring adopts a template based approach. The role of template is to wrap up all the boiler plate code and provide interfaces so that the developers can focus on business specific code.Spring handles the data access using templates and callbacks.Any data access technology is used it has a fixed part(boiler plate code) and a programmer defined part. The fixed part like opening closing connection is handled by template and the variable part like how to handle the result is handled by callbacks.
Spring comes with several data access template for different persistence mechanism:

  • JdbcTemplate
  • NamedParameterJdbcTemplate
  • SimpleJdbcTemplate
  • HibernateTemplate
  • JpaTemplate
At the moment we are looking into JDBC way of interacting to database so Let's see how templates related to JDBC come to action. In this regard we will use JdbcTemplate first.JdbcTemplate class simplifies working with JDBC. It  automatically handles resource management, exception handling and transaction management.It is a thread safe class so you can use a single instance that many classes can use. Underlying Connection can also be accessed.So now our implementation class for JdbcDao looks like

StudentJdbcDao.java

public class StudentJdbcDao implements StudentDao{

   private JdbcTemplate jdbcTemplate;
   
   public void setJdbcTemplate(JdbcTemplate jdbcTemplate){
        this.jdbcTemplate = jdbcTemplate;
   }

   public void saveStudent(Student student) {
        jdbcTemplate.update
        ("insert into STUDENT (name) values (?)",new Object[] {student.getName()} );
   }
}

Now the question is how StudentJdbcDao gets the jdbcTemplate. In true Spring fashion let's wire the relationship.

In context.xml

<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
    <property name="dataSource" ref="dataSource" />
</bean>

<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource“
    destroy-method="close">
  <property name="driverClassName" value="org.hsqldb.jdbcDriver" />
  <property name="url" value="jdbc:hsqldb:hsql://localhost" />
  <property name="username" value="sa" />
  <property name="password" value="" />
</bean>

dataSource is the bean in which we provide all the database connection properties. Than we inject dataSource in jdbcTemplate and than let's inject jdbcTemplate in StudentJdbcDao.

<bean id="studentDao" class="StudentJdbcDao">
   <property name="jdbcTemplate" ref="jdbcTemplate" />
</bean>

Note that we have given the id as studentDao but the actual class is StudentJdbcDao(Use full package name). We can now access our dao in calling code:

Student student = new Student();
student.setName("AmitabhDao");
StudentDao studentDao = (StudentDao)appContext.getBean("studentDao");
studentDao.saveStudent(student);

It's possible that we might have many dao classes to handle different database interactions. So that will result in writing a lot of code to handle jdbcTemplate. To solve this Spring comes with a convenient class JdbcDaoSupport which gives access to JdbcTemplate

public class StudentJdbcDao extends JdbcDaoSupport implements StudentDao {
    
   public void saveStudent(Student student) {
    this.getJdbcTemplate().update
        ("insert into STUDENT (name) values (?)",
                     new Object[] {student.getName()} );        
  }
}

The configuration code in context.xml remains same.

Configuring Data Sources

Spring provides a number of ways to configure data source.
  • Using JDBC driver
  • JNDI Lookup
  • Pool connections

We saw earlier how to handle the connection using connection properties when we are working with JDBC directly. Spring provides two basic JDBC driver based datasource

DriverManagerDataSource –  A new connection is returned every time. Connections are not pooled.
SingleConnectionDataSource – It returns always the same connection. It acts like a pool of one connection.

<bean id =“datasource”   class=“org.springframework.jdbc.datasource.DriverManagerDataSource”>
   <property name="driverClassName“ value="org.hsqldb.jdbcDriver" />
   <property name="url" value="jdbc:hsqldb:hsql://localhost" />
   <property name="username" value="sa" />
   <property name="password" value="" />
</bean>

For JNDI datasource, the datasource is feteched using JNDI lookup

<bean id ="datasource" class="org.springframework.jndi.JndiObjectFactoryBean">
   <property name=“jndiName” value=/jdbc/ds”/>
   <property name=“resourceRef” value=“true”/>
</bean>

With jee namespace, it can be written more succinctly

<jee:jndi-lookup id=“datasource” jndi-name=“jdbc/ds” resource-ref=“true” />

Spring provides the capability to pool the datasource in the application itself using Jakarta’s Commons Database Connection Pools (DBCP)

<bean id ="datasource" class="org.apache.commons.dbcp.BasicDataSource">
   <property name="driverClassName" value="org.hsqldb.jdbcDriver" />
   <property name="url" value="jdbc:hsqldb:hsql://localhost" />
   <property name="username" value="sa" />
   <property name="password" value="" />
   <property name=“initialSize” value=“5”/>
   <property name=“maxActive” value=“10”/>
</bean>

JDBCTemplate

Spring provides three types of template class to work with JDBC
  • JdbcTemplate: This is the way we did earlier. It's the basic way of doing the database access. The parameters to SQL are passed as indexes parameters.
  • NamedParameterJdbcTemplate: The parameters are send as named value pair in a Map.
  • SimpleJdbcTemplate: It uses features like autoboxing and ((Generics)). Brings type safety and casting need not be done.

Each template classs has supporting DaoSupport class for easy wiring of templates.We saw the usage of JdbcTemplate earlier. Let's look how NamedParameterJdbcTemplate works. In this case the dao class looks like

public class StudentJdbcDao implements StudentDao {
    
    protected NamedParameterJdbcTemplate namedJdbcTemplate;

    //Method to inject NamedParameterJdbcTemplate   
    public void setNamedJdbcTemplate (NamedParameterJdbcTemplate namedJdbcTemplate) {
    this.namedJdbcTemplate = namedJdbcTemplate;
    }

    public void saveStudent(Student student) {
    Map parameters = new HashMap();
    parameters.put("name",student.getName());
    namedJdbcTemplate.update
        ("insert into STUDENT (name) values (:name)",
          parameters);        
    }
...
}

In configuration XML

<bean id="jdbcTemplate" class="org.springframework.jdbc.core.simple.SimpleJdbcTemplate">
    <constructor-arg ref="dataSource" />
</bean>

Note that the datasource is injected as constructor argument.

In case of SimpleJdbcTemplate the Dao class looks like:

protected SimpleJdbcTemplate simpleJdbcTemplate;

    //Setter method for SimpleJdbcTemplate
    ...

    public void saveStudent(Student student) {
         //Note how the arguments are passed. It uses varargs feature.
    simpleJdbcTemplate.update
        ("insert into STUDENT (name,percentage) values (?,?)",
         student.getName(),student.getPercentage());

Configuration XML

<bean id="jdbcTemplate" class="org.springframework.jdbc.core.simple.SimpleJdbcTemplate">
    <constructor-arg ref="dataSource" />
</bean>

Fetching Collections

Now let's look how we can fetch the collection from database. We will use the SimpleJdbcTemplate to do that

public List<Student> getAllStudents() {
   return simpleJdbcTemplate.query
    ("Select name as Name from Student",
               new ParameterizedRowMapper<Student>(){
              public Student mapRow(ResultSet rs,int rowNum)
                  throws SQLException {
                    Student student = new Student();
                student.setName(rs.getString("Name"));
                return student;                      
          }
        }        
   );
}

Note how anonymous inner class ParameterizedRowMapper<Student> is registered as a parameter to the query method. Now to understand that again think in terms of moving part and fixed part. The moving part is the query which is passed as first argument. The fixed part is the boiler plate code which handles opening and closing connection, making and executing statement and iterating through the result. Again the moving part is how to handle each iteration. This is provided by the mapRow method. The mapRow method is called as a callback for every iteration.

Transactions

Spring supports the following way of defining transaction
  • Declarative
  • Programmatic

Spring has no capability to manage transactions directly. The transaction management is delegated to platform specific transaction implementation provided by either JTA or the persistence mechanism.Spring supports wide variety of transaction manager.
Plain JDBC transaction is handled by DataSourceTransactionManager.DataSourceTransactionManager manages the transaction automatically by calling commit on success and in the case of failure calling the rollback.Define the transaction manager and transaction template

<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
   <property name="dataSource" ref="dataSource" />
</bean>

<bean id="transactionTemplate" class="org.springframework.transaction.support.TransactionTemplate" >
   <property name="transactionManager" ref="transactionManager"/>
</bean>

Controlling transaction programmatically

public void saveUser(final User user) {
   transactionTemplate.execute(
    new TransactionCallback(){
    public Object doInTransaction(TransactionStatus ts){
        try{
                     userDao.saveUser(user);
                }catch(Exception ex){
                  ts.setRollbackOnly();
                }
    return null;
});}

Spring defines declarative transaction using attributes:

  • Propagation behavior - This tells how the transaction boundaries need to be handled.
  • Isolation level - The isolation level decides how the common data between two transactions are shared.
  • Read-only - This is an optimization hint to the database. The database can optimize if it comes to know that the particular sql call is a read only call. For example fetching the list of students todisplay only. Not all database support this feature.
  • Transaction timeout - For long running transaction the timeout tells the time limit to which the transaction can run. After timeout the transaction is aborted by the database.
  • Rollback Rules -Tell which exception will result in a rollback.By default transactions are rolled back only on runtime exceptions and not on checked exceptions.In Spring, the behaviour can be changed by doing rollback on checked exception and not to do rollback on unchecked exceptions.

Annotation Driven Transaction

With annotations introduced in Java 5+, handling transaction has been made very easy. Add the transaction manager by registering the transaction manager in configuration XML. This also has been made easy by using tx namespace

<tx:annotation-driven/>

Add the annotation

@Transactional(propagation=Propagation.SUPPORTS, readOnly=true)
public class UserListService {

   @Transactional(propagation=Propagation.REQUIRED, readOnly=false)
   public void insertUser(final User user) {
      userDao.saveUser(user);
   }

The annotation drives the behavior for all methods in the class. The behavior for individual method can be overridden by putting annotation on individual method.

More Articles on Spring

Tuesday, August 12, 2014

Export Postgres Database Table Data to File

Many times we need the data from tables in file formats. It could be a plain text file or a csv file. Let's look into various ways of exporting the data out of postgres table.

Export Postgres table to csv

csv stands for comma separated file and is a very popular format for opening the file in excel. To export csv we can follow one of the following ways:


  • Let's assume we have a Student table in the database. So to export this table data with header in csv file, enter into psql console and issue the following command. Assuming the file will be generated at D drive. For Linux you can change the file path accordingly.
             copy Student to 'D:\student.csv' csv header
       
         A file with student.csv will be created

  • If you want certain columns only for Student table, then issue the following command from command prompt
              psql -U <db user name> <db_name> -F , --no-align -c "SELECT id,name from Student" > student.csv

  • For tab delimited text file
      psql -U <db user name> <db_name> -F $'\t' --no-align -c "SELECT id,name from Student" > studentwithTabs.txt

Export Postgres table to File

Enter into psql console. In the console, issue the following command

             \o output.txt

Now whatever command you will fire, the result will go into output.txt file.  Giving the \o command without file name will start putting the output back into console.