Parsing log files (cygwin grep.... awk...)

[hurtlocker]

Hello.... I am trying to create reports off of backup log files - using cygwin (bash-> grep, awk) on WinServer2008R2/backup4all-Pro-4.6 However the log files appear to have zero fills out to 16bits so grep awk does not work. Things like bash-> cat, tail, work. All windows stuff like wordpad, textpad even notepad work. I can even copy paste from the cygwin rxvt window and vi paste and the zeros get stripped. But I am trying to script. Here is a cygwin tail list of the file followed by a octel dump of characters (od -c): (randy.log is a copy)

Administrator@resucsdapp6 /cygdrive/c/users/administrator/appdata/Roaming/Softland/Backup4all Professional 4/Logs
$ tail -2 randy.log
Backup completed at 1/3/2012 9:12:17 PM with 0 error(s), 1 warning(s)

Administrator@resucsdapp6 /cygdrive/c/users/administrator/appdata/Roaming/Softland/Backup4all Professional 4/Logs
$ od -c randy.log
0024260 \0 B \0 a \0 c \0 k \0 u \0 p \0 \0 <- (\0 means all zero bits is those 8 bits)
0024300 c \0 o \0 m \0 p \0 l \0 e \0 t \0 e \0
0024320 d \0 \0 a \0 t \0 \0 1 \0 / \0 3 \0
0024340 / \0 2 \0 0 \0 1 \0 2 \0 \0 9 \0 : \0
0024360 1 \0 2 \0 : \0 1 \0 7 \0 \0 P \0 M \0
0024400 \0 w \0 i \0 t \0 h \0 \0 0 \0 \0
0024420 e \0 r \0 r \0 o \0 r \0 ( \0 s \0 ) \0
0024440 , \0 \0 1 \0 \0 w \0 a \0 r \0 n \0
0024460 i \0 n \0 g \0 ( \0 s \0 ) \0 \r \0 \n \0
0024500

I thought maybe it is 16bit unicode so I tried piconv and even copied to real linux and tried iconv
But as you can see the results are not as I was hoping for.
root@resucsdapp1 ~]# iconv -f ISO-8859-1 -t UTF-8 randy.log -o randyconv.log
[root@resucsdapp1 ~]# ls -ld *ran*
-rw-r--r-- 1 root root 7494 Jan 3 17:26 randyconv.log
-rwx------ 1 root root 7490 Jan 3 17:23 randy.log
[root@resucsdapp1 ~]#

I will keep studying but maybe someone can give some pointers to help out on this?

Thank you..very much..

[hurtlocker]

SOLUTION:
Hello.... using a different "from type" worked ***UTF16***
[root@resucsdapp1 ~]# iconv -f UTF16 -t UTF8 randy.log -o randyconv.log

[root@resucsdapp1 ~]# ll *ran*
-rw-r--r-- 1 root root 5281 Jan 4 08:57 randyconv.log <- half the size
-rwx------ 1 root root 10560 Jan 4 08:56 randy.log

[root@resucsdapp1 ~]# od -c randyconv.log | tail -7
0012120 c k u p : \r \n B a c k u p <- no more \0 zero fills
0012140 c o m p l e t e d a t 1 /
0012160 3 / 2 0 1 2 9 : 1 2 : 1 7 P
0012200 M w i t h 0 e r r o r ( s
0012220 ) , 1 w a r n i n g ( s ) \r
0012240 \n
0012241
[root@resucsdapp1 ~]#

Borrowed a real linux for iconv - Now I just need to find iconv for cygwin (not easy)

Hope this helps someone someday

[hurtlocker]

LAST POST on Subject:
If you just search iconv in the cygwin install gui, it will pull only libiconv2 as iconv() function and you'll think you have to go down the compile route (ugh)

To get iconv for cygwin you need to go to libs and select libiconv

syntax a bit different from linux
$ iconv -f UTF-16 -t UTF-8 randy.log > randy8.log
$ ll *ran*
-rwx------+ 1 Administrator Domain Users 10560 Jan 4 07:12 randy.log
-rw-r--r--+ 1 Administrator Domain Users 5278 Jan 4 10:19 randy8.log

Didn't mean to turn this in to a cygwin thread but for those who want to write summary reports for many backup4all backups using their logs and you are a UTF-8 programmer this is very powerful

THANK YOU ! - complete


BTW: useful command:
$ cygcheck -p iconv.exe
Found 4 matches for iconv.exe
libiconv/libiconv-1.13.1-1 GNU character set conversion library and utilities
libiconv/libiconv-1.13.1-2 GNU character set conversion library and utilities
libiconv/libiconv-1.14-1 GNU character set conversion library and utilities
libiconv/libiconv-1.14-2 GNU character set conversion library and utilities