Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
create a file from data in multiple files/directories
#1
I have a bunch of log files and would like to extract the important lines of data using regex or something similar, and compile it in a csv with names of the parent folder.

The part I'm stuck with is this-how do I start?
How do I create the folder selection menu?
How do I create a new file to write to?
how do I go through each folder file-by-file and line-by-line and only copy the lines that match the criteria ?
How do I get the name of the parent folder?
How do I make sure that it doesn't get stuck? (are there any ways to multi/green thread the function if there are too many files)

I'll be spending some time on the manual in the future to figure this out so while I don't expect any help, it's greatly appreciated.

(Looks like you can't have a powerful tool and have it be quick+easy to learn at the same time.)
#2
1.
Quote:The part I'm stuck with is this-how do I start?

Start from 'Enumerate files' dialog, which is in the floating toolbar. It inserts some code. Edit it. Example:
Macro Macro1118
Code:
Copy      Help
out

Dir d
foreach(d "C:\*" FE_Dir 0x6)
,str sPath=d.FileName(1)
,if(d.FileAttributes&FILE_ATTRIBUTE_DIRECTORY)
,,;out "-------- folder: %s" sPath
,else
,,sel sPath 3
,,,case "*\$Recycle.Bin\*" continue
,,,case ["*.log","*.txt"]
,,,case else continue
,,out sPath
,,;long size=d.FileSize
,,;str sData.getfile(d.FileName(1));; err out "cannot open: %s" sPath; continue

Or use function GetFilesInFolder. Easier but less flexible, eg you cannot see progress.

2.
Quote:How do I create the folder selection menu?
To display standard folder selection dialog, use function BrowseForFolder. More difficult would be to select multiple folders.

3.
Quote:How do I create a new file to write to?
If need CSV format, use ICsv interface. Add rows, and call function ToFile. It also supports appending, in case if will be too big for memory.

4.
Quote:how do I go through each folder file-by-file and line-by-line and only copy the lines that match the criteria ?
See #1. To get line-by-line, use foreach.

str s
foreach s sData
,out s

5.
Quote:How do I get the name of the parent folder?
str sFolder.getpath(sPath "")

6.
Quote:How do I make sure that it doesn't get stuck?
Use out to display paths in QM output. Press Pause to stop.
#3
thanks a lot for your help

I have two additional questions

how do I get the folder name instead of the folder path
and how to I I find files across multiple directories

this is what I have sofar
Code:
Copy      Help
str s
if(!BrowseForFolder(s)) ret
out s

ICsv x=CreateCsv(1)
x.ColumnCount=4

str f
str _ug
str _ugDate
str _ugTime
str _line
str _sData
str _patternug="ug::[^\n]*"
str _patternDate="\d\d\d\d-\d\d-\d\d"
str _patternTime="\d\d:\d\d:\d\d"

foreach f s
,f+iif(f.end("\") "*" "\*")
,Dir d
,foreach(d f FE_Dir 0x6)
,,str _sPath=d.FileName(1)
,,_sData.getfile(_sPath)
,,str _sFolder.getpath(_sPath "")
,,out _sPath
,,foreach _line _sData
,,,if(findrx(_line _patternug 0 2 _ug)>=0)
,,,,findrx(_line _patternDate 0 2 _ugDate)
,,,,findrx(_line _patternTime 0 2 _ugTime)
,,,,;add empty row and use Cell
,,,,int r=x.AddRowMS(-1)
,,,,x.Cell(r 0)=_sFolder; x.Cell(r 1)=_ugDate; x.Cell(r 2)=_ugTime; x.Cell(r 3)=_ug

out x
str _filename = "$desktop$\data.csv"
x.ToFile(_filename)
#4
Macro Macro1495
Code:
Copy      Help
str _sFolder="c:\f1\f2"
str folderName.getfilename(_sFolder)
out folderName
#5
Quote:how to I I find files across multiple directories

Instead of BrowseForFolder, make list of folders in macro.

Code:
Copy      Help
s=
;c:\folder1
;c:\folder2
#6
Simple dialog for selecting folders.

Function DialogSelectFilesFolders
Code:
Copy      Help
;/
function! str&s [hwndOwner]

;Shows simple dialog where you can enter paths of files or folders.
;You can use hotkey to select files from Windows Explorer.
;Returns 1 on OK, 0 on Cancel.

;s - variable that receives paths.
;hwndOwner - owner window handle.


;EXAMPLE
;str sFiles sFile
;if(!DialogSelectFilesFolders(sFiles)) ret
;foreach sFile sFiles
,;out sFile


s=""
str controls = "3"
str e3
if(!ShowDialog("DSFF_Dialog" &DSFF_Dialog &controls)) ret
s=e3
ret 1

Function DSFF_Dialog
Code:
Copy      Help
;\Dialog_Editor
function# hDlg message wParam lParam
if(hDlg) goto messages

;BEGIN DIALOG
;0 "" 0x90C80AC8 0x0 0 0 275 190 "Select Files and Folders"
;4 Static 0x54000000 0x0 6 6 264 36 "Enter one or more paths of files or folders.[][]You can select one or more files or folders in Explorer, and press F11.[]Repeat with other files/folders, if need."
;3 Edit 0x54231044 0x200 6 44 264 122 ""
;1 Button 0x54030001 0x4 90 172 48 14 "OK"
;2 Button 0x54030000 0x4 142 172 48 14 "Cancel"
;END DIALOG
;DIALOG EDITOR: "" 0x2030208 "" "" ""

ret
;messages
sel message
,case WM_INITDIALOG
,if(!GetParent(hDlg)) ont hDlg
,__RegisterHotKey-- hk.Register(hDlg 1 0 VK_F11)
,
,case WM_DESTROY
,hk.Unregister
,
,case WM_HOTKEY
,int hwnd=win()
,;Get ShellBrowserWindow interface by enumerating shell windows
,SHDocVw.ShellWindows sw._create
,SHDocVw.ShellBrowserWindow b
,foreach(b sw)
,,int hwnd2=b.HWND; err continue
,,if(hwnd=hwnd2) goto g1
,ret
,;g1
,str s.getwintext(id(3 hDlg))
,;get shell folder view document
,Shell32.ShellFolderView sfw=b.Document; err ret
,Shell32.FolderItem fi
,;enumerate selected items
,foreach(fi sfw.SelectedItems) s.addline(fi.Path)
,;add to textbox
,EditReplaceSel hDlg 3 s 1
,
,case WM_COMMAND goto messages2
ret
;messages2
sel wParam
,case IDOK
,case IDCANCEL
ret 1
#7
ačiū

turns out the problem I was having was that I couldn't parse the directories as files because I was missing this conditional
if(d.FileAttributes&FILE_ATTRIBUTE_DIRECTORY==0)
#8
foreach FE_Dir with flag 0x6 retrieves files and folders. If need only files, use flags 0x4.


Forum Jump:


Users browsing this thread: 1 Guest(s)