I have a bunch of files in a folder. They start from 20140211, and go until the present. Every weekday a new file gets dropped in this folder. Every day I run a batch process to pick up the previous days file, by date, and Bulk Insert it into a staging table and then I load date from that staging table into my production table. There are 6 parts: delete all data from the staging table [SPGT_Daily], write data from the file in the folder (one day before today) to a table, and then copy the certain data points (when [Index Code]='SPGTINFR' & when [Index Code] ='SPGTINTR') from the staging table to the production table [GII_ICL]. The next 3 parts are just like the first three, but the changes are mode to different tables.
EVERY SINGLE PART WORKS FINE, BUT ALL TOGETHER THE SPROC DOESN'T WORK. For one thing, if I highlight the following script and hit F5, it gets the previous day’s data.
declare @fullpath1varchar(1000)
select @fullpath1='''\\local\mx\london\FTP\'+convert(varchar,getdate()-1, 112)+'_SPGT.SPL'''
declare @cmd1nvarchar(1000)
print(@cmd1)
select @cmd1='bulk insert [dbo].[SPGT_Daily] from '+ @fullpath1 +' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 223, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec(@cmd1)
If I run the whole SProc, it pulls in all the data ALL the files in the folder. That makes no sense to me!! Everything gets messed up after this. Can someone here please give me a little guidance here? I must be missing something really basic.
Here’s the entire SProc script:
USE [London_Data]
GO
SETANSI_NULLSON
GO
SETQUOTED_IDENTIFIERON
GO
ALTERPROCEDURE [dbo].[LoadDaily]
AS
BEGIN
-------------------------------------------
----PART1
delete
from [dbo].[SPGT_Daily]
-------------------------------------------
----PART2
declare @fullpath1varchar(1000)
select @fullpath1='''\\local\mx\london\FTP\'+convert(varchar,getdate()-1, 112)+'_SPGT.SPL'''
declare @cmd1nvarchar(1000)
print(@cmd1)
select @cmd1='bulk insert [dbo].[SPGT_Daily] from '+ @fullpath1 +' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 223, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec(@cmd1)
-------------------------------------------
----PART3
INSERTinto GII_ICL(Date, Index_Code, Price_Return_Level, Gross_Total_Return,Description)
SELECT [DATE OF INDEX],
'SPGTINFR',
max(casewhen [Index Code] ='SPGTINFR'then [Index Value]end),
max(casewhen [Index Code] ='SPGTINTR'then [Index Value]end),
max(casewhen [Index Code] ='SPGTINFR'then [Index Name]end)
from SPGT_Daily
where [Index Code] in('SPGTINFR','SPGTINTR')
groupby [DATE OF INDEX];
-------------------------------------------
----PART4
delete
from [dbo].[SPBMI_Daily]
-------------------------------------------
----PART5
declare @fullpath2varchar(1000)
select @fullpath2='''\\local\mx\london\FTP\'+convert(varchar,getdate()-1, 112)+'_SPBMI_GL_PROP_USD_C.SPL'''
declare @cmd2nvarchar(1000)
print(@cmd2)
select @cmd2='bulk insert [dbo].[SPBMI_Daily] from '+ @fullpath2 +' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 3447, ROWTERMINATOR=''0x0a'')'
print(@cmd2)
exec(@cmd2)
-------------------------------------------
----PART6
INSERTinto WPI_ICL(Date, Index_Code, Price_Return_Level, Gross_Total_Return,Description)
SELECT [DATE OF INDEX],
'SPCBMIRWDPROPUSD',
max(casewhen [Index Code] ='SPCBMIRWDPROPUSD'then [Index Value]end),
max(casewhen [Index Code] ='SPCBMIRWDPROPUSD.TR'then [Index Value]end),
max(casewhen [Index Code] ='SPCBMIRWDPROPUSD'then [Index Name]end)
from SPBMI_Daily
where [Index Code] in('SPCBMIRWDPROPUSD','SPCBMIRWDPROPUSD.TR')
groupby [DATE OF INDEX];
--------------------------------------------
END
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.