Library code snippets

Bulk Insert from Flat File Using Transact-SQL

In a typical IT environment, it is often necessary to import flat files to SQL Server tables.
Sometimes it is necessary to format flat file according to our table structure using delimiters.
In this Code Snippet, I am going to discuss how to import Flat files to SQL Server.

Step 1

Create a Destination table [Customer_Sample] using the following schema.

 

Use the following script to create this table:

if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[Customers_Sample]') and OBJECTPROPERTY(id, N'IsUserTable') = 1)
drop table [dbo].[Customers_Sample]
GO

CREATE TABLE [dbo].[Customers_Sample] (
[CustomerID] [nchar] (5) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
[CompanyName] [nvarchar] (40) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
[ContactName] [nvarchar] (30) COLLATE SQL_Latin1_General_CP1_CI_AS NULL ,
[ContactTitle] [nvarchar] (30) COLLATE SQL_Latin1_General_CP1_CI_AS NULL
) ON [PRIMARY]
GO

The following picture is the screen shot of Source Data file (Flat File)

Step 2

Create Schema File using Notepad.

Schema File Details:

8.0
4
1 SQLCHAR 0 5 "" 1 CustomerID SQL_Latin1_General_Cp437_BIN
2 SQLCHAR 0 40 "" 2 CompanyName SQL_Latin1_General_Cp437_BIN
3 SQLCHAR 0 30 "" 3 ContactName SQL_Latin1_General_Cp437_BIN
4 SQLCHAR 0 30 "\n" 4 ContactTitle SQL_Latin1_General_Cp437_BIN

Save this file with “.FMT” Extension (for example: bcp.fmt)

The structure of the Schema File is explained in the following image.

 

Step 3

Create the following Store Procedure [Text_File_Bulk_Import] to import Text file data to Destination Table [Customer_Sample]

CREATE PROC Text_File_Bulk_Import
@SourceFilePath varchar(50),
@FormatFilePath varchar(50),
@RowNumber int
as
SET ANSI_WARNINGS OFF
DECLARE @str_command nvarchar(150)
SET @str_command = 'BULK INSERT [Customer_Sample] FROM ''' + @SourceFilePath
+ ''' WITH (formatfile = ''' + @FormatFilePath + ''', firstrow =' + cast(@RowNumber as nvarchar) + ')'
EXEC SP_EXECUTESQL @str_command

Execute the Store Procedure [Text_File_Bulk_Import]

Text_File_Bulk_Import 'c:\Flat_file.txt','c:\bcp.fmt', 3
  • 'c:\flat_file.txt' – Source Text File Name
  • 'c:\ bcp.fmt' - Schema File Name
  • 3 - First Row Number (Actual Data starts from 4th row only)

Comments

  1. 22 Apr 2009 at 17:40
    I have an aspx application that is calling a stored procedure that does a bulk insert very similar to what you have. Unfortunately when the datafile is large 72,000 rows it fails and gives me 'Internet Explorer cannot display the webpage'. When I cut the file down to 9,000 rows it works fine. When I go to SqlServer Management Studio and run the proc, passing the parameters it loads all 72,000 rows. Any ideas?
  2. 29 Jun 2006 at 14:19

    Hello,

    I am getting the same error message:
    Server: Msg 4839, Level 16, State 1, Line 1
    Cannot perform bulk insert. Invalid collation name for source column 4 in format file 'c:\bcp.fmt'.

    I tried to change into the bcp file the collation to be SQL_Latin1_General_CP1_CI_AS. Still doesn't work.

    Someone can help me please?

     

    Thanks a lot.

  3. 27 Mar 2006 at 05:25

    I got this error . 'Cannot perform bulk insert. Invalid collation name for source column 4 in format file 'c:\bcp.fmt'.' I copied as it was on the post. Any ideas?



  4. 10 Feb 2006 at 09:42

    Good piece of knowledge imparting code.
    Got to learn something new.

  5. 07 Feb 2006 at 13:43

    Hari,


    Thanx for posting this valuable code snippet.  It was really useful to me.  I learnt a new topic today.  Once again thanx.


    Regards
    Saravanan

  6. 01 Jan 1999 at 00:00

    This thread is for discussions of Bulk Insert from Flat File Using Transact-SQL .

Leave a comment

Sign in or Join us (it's free).

M M Harinath
AddThis

Related podcasts

  • Stack Overflow: Podcast #28

    This is the twenty-eighth episode of the StackOverflow podcast, where Joel and Jeff discuss Windows Azure, SQL Server 2008 full text search, Bayesian filtering, porn detection, and project management — among other things. Jeff met the inestimable Joey DeVilla aka Accordion Guy...

Events coming up

  • Jul 7

    Midlands PASS Chapter July Meeting

    Columbia, United States

    Midlands PASS welcomes SQL Server MVP John Welch. John will be giving a talk on Getting Started with Analysis Services 2008. We meet in the Training Concepts facility (Suite 502).

We'd love to hear what you think! Submit ideas or give us feedback