In SAS its possible to go through a dataset and used lagged values.
The way I would do it is to use a function that does a "lag", but this presumably would produce a wrong value at the beginning of a chunk. For example if a chunk starts at row 200,000, then it will assume an NA for a lagged value that should come instead from row 199,999.
Is there a solution for this?
Matt Parker :
Here's another approach for lagging: self-merging using a shifted date. This is dramatically simpler to code and can lag several variables at once. The downsides are that it takes 2-3 times longer to run than my answer using transformFunc, and requires a second copy of the dataset.\n\n# Get a sample dataset\nsourcePath <- file.path(rxGetOption(\"sampleDataDir\"), \"DJIAdaily.xdf\")\n\n# Set up paths for two copies of it\nxdfPath <- tempfile(fileext = \".xdf\")\nxdfPathShifted <- tempfile(fileext = \".xdf\")\n\n\n# Convert \"Date\" to be Date-classed\nrxDataStep(inData = sourcePath,\n outFile = xdfPath,\n transforms = list(Date = as.Date(Date)),\n overwrite = TRUE\n)\n\n\n# Then make the second copy, but shift all the dates up \n# one (or however much you want to lag)\n# Use varsToKeep to subset to just the date and \n# the variables you want to lag\nrxDataStep(inData = xdfPath,\n outFile = xdfPathShifted,\n varsToKeep = c(\"Date\", \"Open\", \"Close\"),\n transforms = list(Date = as.Date(Date) + 1),\n overwrite = TRUE\n)\n\n# Create an output XDF (or just overwrite xdfPath)\nxdfLagged2 <- tempfile(fileext = \".xdf\")\n\n# Use that incremented date to merge variables back on.\n# duplicateVarExt will automatically tag variables from the \n# second dataset as \"Lagged\".\n# Note that there's no need to sort manually in this one - \n# rxMerge does it automatically.\nrxMerge(inData1 = xdfPath,\n inData2 = xdfPathShifted,\n outFile = xdfLagged2,\n matchVars = \"Date\",\n type = \"left\",\n duplicateVarExt = c(\"\", \"Lagged\")\n)\n",
2015-06-26T13:34:17