Index: branches/Bungo/i386/boot0/boot0.old =================================================================== --- branches/Bungo/i386/boot0/boot0.old (revision 0) +++ branches/Bungo/i386/boot0/boot0.old (revision 2840) @@ -0,0 +1,807 @@ +; Copyright (c) 1999-2003 Apple Computer, Inc. All rights reserved. +; +; @APPLE_LICENSE_HEADER_START@ +; +; Portions Copyright (c) 1999-2003 Apple Computer, Inc. All Rights +; Reserved. This file contains Original Code and/or Modifications of +; Original Code as defined in and that are subject to the Apple Public +; Source License Version 2.0 (the "License"). You may not use this file +; except in compliance with the License. Please obtain a copy of the +; License at http://www.apple.com/publicsource and read it before using +; this file. +; +; The Original Code and all software distributed under the License are +; distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, EITHER +; EXPRESS OR IMPLIED, AND APPLE HEREBY DISCLAIMS ALL SUCH WARRANTIES, +; INCLUDING WITHOUT LIMITATION, ANY WARRANTIES OF MERCHANTABILITY, +; FITNESS FOR A PARTICULAR PURPOSE OR NON- INFRINGEMENT. Please see the +; License for the specific language governing rights and limitations +; under the License. +; +; @APPLE_LICENSE_HEADER_END@ +; +; Boot Loader: boot0 +; +; A small boot sector program written in x86 assembly whose only +; responsibility is to locate the active partition, load the +; partition booter into memory, and jump to the booter's entry point. +; It leaves the boot drive in DL and a pointer to the partition entry in SI. +; +; This boot loader must be placed in the Master Boot Record. +; +; In order to coexist with a fdisk partition table (64 bytes), and +; leave room for a two byte signature (0xAA55) in the end, boot0 is +; restricted to 446 bytes (512 - 64 - 2). If boot0 did not have to +; live in the MBR, then we would have 510 bytes to work with. +; +; boot0 is always loaded by the BIOS or another booter to 0:7C00h. +; +; This code is written for the NASM assembler. +; nasm boot0.s -o boot0 + +; +; This version of boot0 implements hybrid GUID/MBR partition scheme support +; +; Written by Tam‡s Kos‡rszky on 2008-03-10 +; +; Turbo added EFI System Partition boot support +; +; Added KillerJK's switchPass2 modifications +; + +; +; Set to 1 to enable obscure debug messages. +; +DEBUG EQU CONFIG_BOOT0_DEBUG + +; +; Set to 1 to enable verbose mode +; +VERBOSE EQU CONFIG_BOOT0_VERBOSE + +; +; Various constants. +; +kBoot0Segment EQU 0x0000 +kBoot0Stack EQU 0xFFF0 ; boot0 stack pointer +kBoot0LoadAddr EQU 0x7C00 ; boot0 load address +kBoot0RelocAddr EQU 0xE000 ; boot0 relocated address + +kMBRBuffer EQU 0x1000 ; MBR buffer address +kLBA1Buffer EQU 0x1200 ; LBA1 - GPT Partition Table Header buffer address +kGPTABuffer EQU 0x1400 ; GUID Partition Entry Array buffer address + +kPartTableOffset EQU 0x1be +kMBRPartTable EQU kMBRBuffer + kPartTableOffset + +kSectorBytes EQU 512 ; sector size in bytes +kBootSignature EQU 0xAA55 ; boot sector signature +kHFSPSignature EQU 'H+' ; HFS+ volume signature +kHFSPCaseSignature EQU 'HX' ; HFS+ volume case-sensitive signature +kFAT32BootCodeOffset EQU 0x5a ; offset of boot code in FAT32 boot sector +kBoot1FAT32Magic EQU 'BO' ; Magic string to detect our boot1f32 code + + +kGPTSignatureLow EQU 'EFI ' ; GUID Partition Table Header Signature +kGPTSignatureHigh EQU 'PART' +kGUIDLastDwordOffs EQU 12 ; last 4 byte offset of a GUID + +kPartCount EQU 4 ; number of paritions per table +kPartTypeHFS EQU 0xaf ; HFS+ Filesystem type +kPartTypeABHFS EQU 0xab ; Apple_Boot partition +kPartTypePMBR EQU 0xee ; On all GUID Partition Table disks a Protective MBR (PMBR) + ; in LBA 0 (that is, the first block) precedes the + ; GUID Partition Table Header to maintain compatibility + ; with existing tools that do not understand GPT partition structures. + ; The Protective MBR has the same format as a legacy MBR + ; and contains one partition entry with an OSType set to 0xEE + ; reserving the entire space used on the disk by the GPT partitions, + ; including all headers. + +kPartActive EQU 0x80 ; active flag enabled +kPartInactive EQU 0x00 ; active flag disabled +kHFSGUID EQU 0x48465300 ; first 4 bytes of Apple HFS Partition Type GUID. +kAppleGUID EQU 0xACEC4365 ; last 4 bytes of Apple type GUIDs. +kEFISystemGUID EQU 0x3BC93EC9 ; last 4 bytes of EFI System Partition Type GUID: + ; C12A7328-F81F-11D2-BA4B-00A0C93EC93B + +%ifdef FLOPPY +kDriveNumber EQU 0x00 +%else +kDriveNumber EQU 0x80 +%endif + +; +; Format of fdisk partition entry. +; +; The symbol 'part_size' is automatically defined as an `EQU' +; giving the size of the structure. +; + struc part +.bootid resb 1 ; bootable or not +.head resb 1 ; starting head, sector, cylinder +.sect resb 1 ; +.cyl resb 1 ; +.type resb 1 ; partition type +.endhead resb 1 ; ending head, sector, cylinder +.endsect resb 1 ; +.endcyl resb 1 ; +.lba resd 1 ; starting lba +.sectors resd 1 ; size in sectors + endstruc + +; +; Format of GPT Partition Table Header +; + struc gpth +.Signature resb 8 +.Revision resb 4 +.HeaderSize resb 4 +.HeaderCRC32 resb 4 +.Reserved resb 4 +.MyLBA resb 8 +.AlternateLBA resb 8 +.FirstUsableLBA resb 8 +.LastUsableLBA resb 8 +.DiskGUID resb 16 +.PartitionEntryLBA resb 8 +.NumberOfPartitionEntries resb 4 +.SizeOfPartitionEntry resb 4 +.PartitionEntryArrayCRC32 resb 4 + endstruc + +; +; Format of GUID Partition Entry Array +; + struc gpta +.PartitionTypeGUID resb 16 +.UniquePartitionGUID resb 16 +.StartingLBA resb 8 +.EndingLBA resb 8 +.Attributes resb 8 +.PartitionName resb 72 + endstruc + +; +; Macros. +; +%macro DebugCharMacro 1 + mov al, %1 + call print_char +%endmacro + +%macro LogString 1 + mov di, %1 + call log_string +%endmacro + +%if DEBUG +%define DebugChar(x) DebugCharMacro x +%else +%define DebugChar(x) +%endif + +;-------------------------------------------------------------------------- +; Start of text segment. + + SEGMENT .text + + ORG kBoot0RelocAddr + +;-------------------------------------------------------------------------- +; Boot code is loaded at 0:7C00h. +; +start: + ; + ; Set up the stack to grow down from kBoot0Segment:kBoot0Stack. + ; Interrupts should be off while the stack is being manipulated. + ; + cli ; interrupts off + xor ax, ax ; zero ax + mov ss, ax ; ss <- 0 + mov sp, kBoot0Stack ; sp <- top of stack + sti ; reenable interrupts + + mov es, ax ; es <- 0 + mov ds, ax ; ds <- 0 + + ; + ; Relocate boot0 code. + ; + mov si, kBoot0LoadAddr ; si <- source + mov di, kBoot0RelocAddr ; di <- destination + cld ; auto-increment SI and/or DI registers + mov cx, kSectorBytes/2 ; copy 256 words + repnz movsw ; repeat string move (word) operation + + ; + ; Code relocated, jump to start_reloc in relocated location. + ; + jmp kBoot0Segment:start_reloc + +;-------------------------------------------------------------------------- +; Start execution from the relocated location. +; +start_reloc: + + DebugChar('>') + +%if DEBUG + mov al, dl + call print_hex +%endif + + ; + ; Since this code may not always reside in the MBR, always start by + ; loading the MBR to kMBRBuffer and LBA1 to kGPTBuffer. + ; + + xor eax, eax + mov [my_lba], eax ; store LBA sector 0 for read_lba function + mov al, 2 ; load two sectors: MBR and LBA1 + mov bx, kMBRBuffer ; MBR load address + call load + jc error ; MBR load error + + ; + ; Look for the booter partition in the MBR partition table, + ; which is at offset kMBRPartTable. + ; + mov si, kMBRPartTable ; pointer to partition table + call find_boot ; will not return on success + +error: + LogString(boot_error_str) + +hang: + hlt + jmp hang + + +;-------------------------------------------------------------------------- +; Find the active (boot) partition and load the booter from the partition. +; +; Arguments: +; DL = drive number (0x80 + unit number) +; SI = pointer to fdisk partition table. +; +; Clobber list: +; EAX, BX, EBP +; +find_boot: + + ; + ; Check for boot block signature 0xAA55 following the 4 partition + ; entries. + ; + cmp WORD [si + part_size * kPartCount], kBootSignature + jne .exit ; boot signature not found. + + xor bx, bx ; BL will be set to 1 later in case of + ; Protective MBR has been found + + inc bh ; BH = 1. Giving a chance for a second pass + ; to boot an inactive but boot1h aware HFS+ partition + ; by scanning the MBR partition entries again. + +.start_scan: + mov cx, kPartCount ; number of partition entries per table + +.loop: + + ; + ; First scan through the partition table looking for the active + ; partition. + ; +%if DEBUG + mov al, [si + part.type] ; print partition type + call print_hex +%endif + + mov eax, [si + part.lba] ; save starting LBA of current + mov [my_lba], eax ; MBR partition entry for read_lba function + cmp BYTE [si + part.type], 0 ; unused partition? + je .continue ; skip to next entry + cmp BYTE [si + part.type], kPartTypePMBR ; check for Protective MBR + jne .testPass + + mov BYTE [si + part.bootid], kPartInactive ; found Protective MBR + ; clear active flag to make sure this protective + ; partition won't be used as a bootable partition. + mov bl, 1 ; Assume we can deal with GPT but try to scan + ; later if not found any other bootable partitions. + +.testPass: + cmp bh, 1 + jne .Pass2 + +.Pass1: + cmp BYTE [si + part.bootid], kPartActive ; In pass 1 we are walking on the standard path + ; by trying to hop on the active partition. + jne .continue + xor dh, dh ; Argument for loadBootSector to skip HFS+ partition + ; signature check. + jmp .tryToBoot + +.Pass2: + cmp BYTE [si + part.type], kPartTypeHFS ; In pass 2 we're going to find a HFS+ partition + ; equipped with boot1h in its boot record + ; regardless if it's active or not. + jne .continue + mov dh, 1 ; Argument for loadBootSector to check HFS+ partition signature. + + DebugChar('*') + + ; + ; Found boot partition, read boot sector to memory. + ; + +.tryToBoot: + + call loadBootSector + jne .continue + jmp SHORT initBootLoader + +.continue: + add si, BYTE part_size ; advance SI to next partition entry + loop .loop ; loop through all partition entries + + ; + ; Scanned all partitions but not found any with active flag enabled + ; Anyway if we found a protective MBR before we still have a chance + ; for a possible GPT Header at LBA 1 + ; + dec bl + jnz .switchPass2 ; didn't find Protective MBR before + call checkGPT + +.switchPass2: + ; + ; Switching to Pass 2 + ; try to find a boot1h aware HFS+ MBR partition + ; + dec bh + mov si, kMBRPartTable ; set SI to first entry of MBR Partition table + jz .start_scan ; scan again + +.exit: + ret ; Giving up. + + + ; + ; Jump to partition booter. The drive number is already in register DL. + ; SI is pointing to the modified partition entry. + ; +initBootLoader: + +DebugChar('J') + +%if VERBOSE + LogString(done_str) +%endif + + jmp kBoot0LoadAddr + + ; + ; Found Protective MBR Partition Type: 0xEE + ; Check for 'EFI PART' string at the beginning + ; of LBA1 for possible GPT Table Header + ; +checkGPT: + push bx + + mov di, kLBA1Buffer ; address of GUID Partition Table Header + cmp DWORD [di], kGPTSignatureLow ; looking for 'EFI ' + jne .exit ; not found. Giving up. + cmp DWORD [di + 4], kGPTSignatureHigh ; looking for 'PART' + jne .exit ; not found. Giving up indeed. + mov si, di + + ; + ; Loading GUID Partition Table Array + ; + mov eax, [si + gpth.PartitionEntryLBA] ; starting LBA of GPT Array + mov [my_lba], eax ; save starting LBA for read_lba function + mov cx, [si + gpth.NumberOfPartitionEntries] ; number of GUID Partition Array entries + mov bx, [si + gpth.SizeOfPartitionEntry] ; size of GUID Partition Array entry + + push bx ; push size of GUID Partition entry + + ; + ; Calculating number of sectors we need to read for loading a GPT Array + ; +; push dx ; preserve DX (DL = BIOS drive unit number) +; mov ax, cx ; AX * BX = number of entries * size of one entry +; mul bx ; AX = total byte size of GPT Array +; pop dx ; restore DX +; shr ax, 9 ; convert to sectors + + ; + ; ... or: + ; Current GPT Arrays uses 128 partition entries each 128 bytes long + ; 128 entries * 128 bytes long GPT Array entries / 512 bytes per sector = 32 sectors + ; + mov al, 32 ; maximum sector size of GPT Array (hardcoded method) + + mov bx, kGPTABuffer + push bx ; push address of GPT Array + call load ; read GPT Array + pop si ; SI = address of GPT Array + pop bx ; BX = size of GUID Partition Array entry + jc error + + ; + ; Walk through GUID Partition Table Array + ; and load boot record from first available HFS+ partition. + ; + ; If it has boot signature (0xAA55) then jump to it + ; otherwise skip to next partition. + ; + +%if VERBOSE + LogString(gpt_str) +%endif + +.gpt_loop: + + mov eax, [si + gpta.PartitionTypeGUID + kGUIDLastDwordOffs] + + cmp eax, kAppleGUID ; check current GUID Partition for Apple's GUID type + je .gpt_ok + + ; + ; Turbo - also try EFI System Partition + ; + + cmp eax, kEFISystemGUID ; check current GUID Partition for EFI System Partition GUID type + jne .gpt_continue + +.gpt_ok: + ; + ; Found HFS Partition + ; + + mov eax, [si + gpta.StartingLBA] ; load boot sector from StartingLBA + mov [my_lba], eax + mov dh, 1 ; Argument for loadBootSector to check HFS+ partition signature. + call loadBootSector + jne .gpt_continue ; no boot loader signature + + mov si, kMBRPartTable ; fake the current GUID Partition + mov [si + part.lba], eax ; as MBR style partition for boot1h + mov BYTE [si + part.type], kPartTypeHFS ; with HFS+ filesystem type (0xAF) + jmp SHORT initBootLoader + +.gpt_continue: + + add si, bx ; advance SI to next partition entry + loop .gpt_loop ; loop through all partition entries + +.exit: + pop bx + ret ; no more GUID partitions. Giving up. + + +;-------------------------------------------------------------------------- +; loadBootSector - Load boot sector +; +; Arguments: +; DL = drive number (0x80 + unit number) +; DH = 0 skip HFS+ partition signature checking +; 1 enable HFS+ partition signature checking +; [my_lba] = starting LBA. +; +; Returns: +; ZF = 0 if boot sector hasn't kBootSignature +; 1 if boot sector has kBootSignature +; +loadBootSector: + pusha + + mov al, 3 + mov bx, kBoot0LoadAddr + call load + jc error + + or dh, dh + jz .checkBootSignature + +.checkHFSSignature: + +%if VERBOSE + LogString(test_str) +%endif + + ; + ; Looking for HFSPlus ('H+') or HFSPlus case-sensitive ('HX') signature. + ; + mov ax, [kBoot0LoadAddr + 2 * kSectorBytes] + cmp ax, kHFSPSignature ; 'H+' + je .checkBootSignature + cmp ax, kHFSPCaseSignature ; 'HX' + je .checkBootSignature + + ; + ; Looking for boot1f32 magic string. + ; + mov ax, [kBoot0LoadAddr + kFAT32BootCodeOffset] + cmp ax, kBoot1FAT32Magic + jne .exit + +.checkBootSignature: + ; + ; Check for boot block signature 0xAA55 + ; + mov di, bx + cmp WORD [di + kSectorBytes - 2], kBootSignature + +.exit: + + popa + + ret + + +;-------------------------------------------------------------------------- +; load - Load one or more sectors from a partition. +; +; Arguments: +; AL = number of 512-byte sectors to read. +; ES:BX = pointer to where the sectors should be stored. +; DL = drive number (0x80 + unit number) +; [my_lba] = starting LBA. +; +; Returns: +; CF = 0 success +; 1 error +; +load: + push cx + +.ebios: + mov cx, 5 ; load retry count +.ebios_loop: + call read_lba ; use INT13/F42 + jnc .exit + loop .ebios_loop + +.exit: + pop cx + ret + + +;-------------------------------------------------------------------------- +; read_lba - Read sectors from a partition using LBA addressing. +; +; Arguments: +; AL = number of 512-byte sectors to read (valid from 1-127). +; ES:BX = pointer to where the sectors should be stored. +; DL = drive number (0x80 + unit number) +; [my_lba] = starting LBA. +; +; Returns: +; CF = 0 success +; 1 error +; +read_lba: + pushad ; save all registers + mov bp, sp ; save current SP + + ; + ; Create the Disk Address Packet structure for the + ; INT13/F42 (Extended Read Sectors) on the stack. + ; + +; push DWORD 0 ; offset 12, upper 32-bit LBA + push ds ; For sake of saving memory, + push ds ; push DS register, which is 0. + mov ecx, [my_lba] ; offset 8, lower 32-bit LBA + push ecx + push es ; offset 6, memory segment + push bx ; offset 4, memory offset + xor ah, ah ; offset 3, must be 0 + push ax ; offset 2, number of sectors + + ; It pushes 2 bytes with a smaller opcode than if WORD was used + push BYTE 16 ; offset 0-1, packet size + + DebugChar('<') +%if DEBUG + mov eax, ecx + call print_hex +%endif + + ; + ; INT13 Func 42 - Extended Read Sectors + ; + ; Arguments: + ; AH = 0x42 + ; DL = drive number (80h + drive unit) + ; DS:SI = pointer to Disk Address Packet + ; + ; Returns: + ; AH = return status (success is 0) + ; carry = 0 success + ; 1 error + ; + ; Packet offset 2 indicates the number of sectors read + ; successfully. + ; + mov si, sp + mov ah, 0x42 + int 0x13 + + jnc .exit + + DebugChar('R') ; indicate INT13/F42 error + + ; + ; Issue a disk reset on error. + ; Should this be changed to Func 0xD to skip the diskette controller + ; reset? + ; + xor ax, ax ; Func 0 + int 0x13 ; INT 13 + stc ; set carry to indicate error + +.exit: + mov sp, bp ; restore SP + popad + ret + + +;-------------------------------------------------------------------------- +; Write a string with 'boot0: ' prefix to the console. +; +; Arguments: +; ES:DI pointer to a NULL terminated string. +; +; Clobber list: +; DI +; +log_string: + pusha + + push di + mov si, log_title_str + call print_string + + pop si + call print_string + + popa + + ret + + +;-------------------------------------------------------------------------- +; Write a string to the console. +; +; Arguments: +; DS:SI pointer to a NULL terminated string. +; +; Clobber list: +; AX, BX, SI +; +print_string: + mov bx, 1 ; BH=0, BL=1 (blue) + cld ; increment SI after each lodsb call +.loop: + lodsb ; load a byte from DS:SI into AL + cmp al, 0 ; Is it a NULL? + je .exit ; yes, all done + mov ah, 0xE ; INT10 Func 0xE + int 0x10 ; display byte in tty mode + jmp short .loop +.exit: + ret + + +%if DEBUG + +;-------------------------------------------------------------------------- +; Write a ASCII character to the console. +; +; Arguments: +; AL = ASCII character. +; +print_char: + pusha + mov bx, 1 ; BH=0, BL=1 (blue) + mov ah, 0x0e ; bios INT 10, Function 0xE + int 0x10 ; display byte in tty mode + popa + ret + + +;-------------------------------------------------------------------------- +; Write the 4-byte value to the console in hex. +; +; Arguments: +; EAX = Value to be displayed in hex. +; +print_hex: + pushad + mov cx, WORD 4 + bswap eax +.loop: + push ax + ror al, 4 + call print_nibble ; display upper nibble + pop ax + call print_nibble ; display lower nibble + ror eax, 8 + loop .loop + + mov al, 10 ; carriage return + call print_char + mov al, 13 + call print_char + + popad + ret + +print_nibble: + and al, 0x0f + add al, '0' + cmp al, '9' + jna .print_ascii + add al, 'A' - '9' - 1 +.print_ascii: + call print_char + ret + +getc: + pusha + mov ah, 0 + int 0x16 + popa + ret +%endif ;DEBUG + + +;-------------------------------------------------------------------------- +; NULL terminated strings. +; +log_title_str db 10, 13, 'boot0: ', 0 + +%if VERBOSE +gpt_str db 'GPT', 0 +test_str db 'test', 0 +done_str db 'done', 0 +%endif + +boot_error_str db 'error', 0 + +;-------------------------------------------------------------------------- +; Pad the rest of the 512 byte sized booter with zeroes. The last +; two bytes is the mandatory boot sector signature. +; +; If the booter code becomes too large, then nasm will complain +; that the 'times' argument is negative. + +; +; According to EFI specification, maximum boot code size is 440 bytes +; + +; +; XXX - compilation errors with debug enabled (see comment above about nasm) +; Azi: boot0.s:808: error: TIMES value -111 is negative +; boot0.s:811: error: TIMES value -41 is negative +; +pad_boot: + times 440-($-$$) db 0 + +pad_table_and_sig: + times 510-($-$$) db 0 + dw kBootSignature + + + ABSOLUTE 0xE400 + +; +; In memory variables. +; +my_lba resd 1 ; Starting LBA for read_lba function + +; END Index: branches/Bungo/i386/boot1/boot1x.s =================================================================== --- branches/Bungo/i386/boot1/boot1x.s (revision 0) +++ branches/Bungo/i386/boot1/boot1x.s (revision 2840) @@ -0,0 +1,642 @@ +; +; Copyright (c) 2014 Zenith432 All rights reserved. +; +; Partition Boot Loader: boot1x +; This version of boot1x tries to find a stage2 boot file in the root folder. +; +; Credits: +; Portions based on boot1f32. +; Thanks to Robert Shullich for +; "Reverse Engineering the Microsoft exFAT File System" dated Dec 1, 2009. +; T13 Commitee document EDD-4 for information about BIOS int 0x13. +; +; This program is designed to reside in blocks 0 - 1 of an exFAT partition. +; It expects that the MBR has left the drive number in DL. +; +; This version requires a BIOS with EBIOS (LBA) support. +; +; This code is written for the NASM assembler. +; nasm -f bin -o boot1x boot1x.s +; +; Written by zenith432 during November 2014. +; + bits 16 + +%define VERBOSE 1 +%define USESIDL 1 +%define USEBP 1 +kMaxBlockCount equ 127 ; Max block count supported by Int 0x13, function 0x42, old school +kBootBlockBytes equ 512 ; Bytes in a exFAT Boot block +kBootSignature equ 0xaa55 ; Boot block signature +kBoot1StackAddress equ 0xfff0 ; Address of top-of-stack 0:0xfff0 +kBoot1LoadAddr equ 0x7c00 ; Address of loaded boot block 0:0x7c00 +kBoot2Segment equ 0x2000 ; Address for boot2 0x2000:0x200 +kBoot2Address equ 512 +kFATBuf equ 0x6c00 ; Address for FAT block buffer 0:0x6c00 (4K space) +kRootDirBuf equ 0x5c00 ; Address for Root Directory block buffer 0:0x5c00 (4K space) +kMaxCluster equ 0xfffffff7 ; exFAT max cluster value + 1 (for FAT32 it's 0x0ffffff8) +kMaxContigClusters equ 1024 ; Max contiguous clusters returned by getRange +kBootNameHash equ 0xdc36 ; exFAT name hash for 'BOOT' (in UTF16LE) +kBoot2MaxBytes equ (512 * 1024 - 512) ; must fit between 0x20200 and 0xa0000 + + struc PartitionEntry ; MBR partition entry (truncated) + times 8 resb 1 +.lba: resd 1 ; starting lba + endstruc + + struc BootParams ; BOOT file parameters +.cluster: resd 1 ; 1st cluster of BOOT +.size: resd 1 ; size of BOOT in bytes + resw 1 +.flag: resb 1 + endstruc + + struc DirIterator ; exFAT Directory Iterator +.entries_end: resb 1 ; beyond last 32-byte entry (possible values 16, 32, 64, 128) +.cluster: resd 1 ; current cluster +.lba_high: resd 1 ; upper 32 bits of lba +.lba_end: resd 1 ; beyond last block (lower 32-bits) +.lba: resd 1 ; current block +.entry: resb 1 ; current 32-byte entry + endstruc + + struc FATCache ; Manages cache state for FAT blocks +.shift: resb 1 ; right shift for converting cluster # to FAT block address +.mask: resw 1 ; bit mask for finding cluster # in FAT block +.lba: resd 1 ; lba # cached in FAT block buffer (note that FAT block address is limited to 32 bits) + endstruc + +%ifdef USEBP +%define BPR bp - gPartitionOffset + +%else +%define BPR +%endif + + section .text + org kBoot1LoadAddr + jmp start + times (3 - $ + $$) nop +gOEMName: times 8 db 0 ; 'EXFAT ' + +; +; Scratch Area +; Used for data structures +; + times (64 - BootParams_size - DirIterator_size - FATCache_size - $ + $$) db 0 +gsParams: times BootParams_size db 0 +gsIterator: times DirIterator_size db 0 +gsFATCache: times FATCache_size db 0 + +; +; exFAT BPB +; +gPartitionOffset: dd 0, 0 +gVolumeLength: dd 0, 0 +gFATOffset: dd 0 +gFATLength: dd 0 +gClusterHeapOffset: dd 0 +gClusterCount: dd 0 +gRootDirectory1stCluster: dd 0 +gVolumeSerialNubmer: dd 0 +gFileSystemRevision: dw 0 ; 0x100 +gVolumeFlags: dw 0 +gBytesPerBlock: db 0 ; range 9 - 12 (power of 2) +gBlocksPerCluster: db 0 ; gBytesPerBlock + gBlocksPerCluster <= 25 (power of 2) +gNumberOfFATs: db 0 ; should be 1 +gDriveSelect: db 0 ; probably 0x80 +gPercentInUse: db 0 + times 7 db 0 +start: + cli + xor eax, eax + mov ss, ax + mov sp, kBoot1StackAddress + sti + mov ds, ax + mov es, ax + + ; + ; Initializing global variables. + ; +%ifdef USEBP + mov bp, gPartitionOffset +%endif +%ifdef USESIDL + ; + ; Shouldn't be necessary to use DS:SI because + ; 1) Existing gPartitionOffset must be correct in + ; order for filesystem to work well when mounted. + ; 2) LBA may be 64 bits if booted from GPT. + ; 3) Not all MBR boot records pass DS:SI + ; pointing to MBR partition entry. + ; +%if 0 + mov ecx, [si + PartitionEntry.lba] + mov [BPR gPartitionOffset + 4], eax + mov [BPR gPartitionOffset], ecx +%endif + ; + ; However, by convention BIOS passes boot + ; drive number in dl, so use that instead + ; of existing gDriveSelect + ; + mov [BPR gDriveSelect], dl +%endif + + ; + ; Initialize FAT Cache + ; + dec eax + mov dword [BPR gsFATCache + FATCache.lba], eax ; alternatively store gFATLength here + mov cl, [BPR gBytesPerBlock] + sub cl, 2 ; range 7 - 10 + mov [BPR gsFATCache + FATCache.shift], cl + neg ax + shl ax, cl + dec ax + mov [BPR gsFATCache + FATCache.mask], ax + + ; + ; Initialize Iterator + ; + mov al, 1 + sub cl, 3 ; range 4 - 7 + shl al, cl + mov [BPR gsIterator + DirIterator.entries_end], al + mov [BPR gsIterator + DirIterator.entry], al + xor eax, eax + mov ecx, [BPR gRootDirectory1stCluster] + mov [BPR gsIterator + DirIterator.lba_end], eax + mov [BPR gsIterator + DirIterator.lba], eax + mov [BPR gsIterator + DirIterator.cluster], ecx + +%ifdef VERBOSE + mov di, init_str + call log_string +%endif + + ; + ; Search root directory for BOOT + ; +.loop: + call nextDirEntry + jc error + cld + lodsb +.revert: + test al, al ; end of root directory? + jz error + cmp al, 0x85 ; file/subdir entry? + jnz .loop + lodsb + cmp al, 2 ; 2ndary count should be 2 + jb .loop + add si, 2 ; skip checksum + lodsb + test al, 0x10 ; file attributes - check not a directory + jnz .loop + call nextDirEntry + jc error + cld + lodsb + cmp al, 0xc0 ; stream extension entry? + jnz .revert + lodsb + mov dl, al ; General 2ndary flag + inc si + lodsb + cmp al, 4 ; name length + jnz .loop + lodsw ; name hash + cmp ax, kBootNameHash + jnz .loop + add si, 2 + mov eax, [si + 4] ; high 32 bits of valid data length + test eax, eax + jz .more + and dl, 0xfe ; if size too big, mark as no allocation +.more: + lodsd ; valid data length + mov [BPR gsParams + BootParams.size], eax + add si, 8 + lodsd ; first cluster + mov [BPR gsParams + BootParams.cluster], eax + mov [BPR gsParams + BootParams.flag], dl + call nextDirEntry + jc error + cld + lodsb + cmp al, 0xc1 + jnz .revert + inc si ; skip flags + lodsd ; unicode chars 1 - 2 + or eax, 0x200020 ; tolower + cmp eax, 0x6f0062 ; 'bo' in UTF16LE + jnz .loop + lodsd ; unicode chars 3 - 4 + or eax, 0x200020 ; tolower + cmp eax, 0x74006f ; 'ot' in UTF16LE + jnz .loop + ; + ; done - found boot file! + ; + mov dl, [BPR gsParams + BootParams.flag] + test dl, 1 ; no allocation or length too big? + jz error + mov ebx, [BPR gsParams + BootParams.size] + cmp ebx, kBoot2MaxBytes + 1 + jnb error + call BytesToBlocks ; convert size to blocks + ; boot2 file size in blocks is in bx +load_boot2: ; anchor for localizing next labels + xor esi, esi ; no blocks after 1st range + test dl, 2 ; FAT Chain? + cmovnz edx, [BPR gsParams + BootParams.cluster] ; if not + jnz .oneshot ; load contiguous file + ; + ; load via FAT + ; + mov si, bx ; total blocks to si +.loop: + mov eax, [BPR gsParams + BootParams.cluster] + mov edx, eax + call getRange + test ebx, ebx + jnz .nonempty + test si, si + jnz error + jmp boot2 +.nonempty: + cmp ebx, esi + cmovnb bx, si + sub si, bx + mov [BPR gsParams + BootParams.cluster], eax +.oneshot: + call ClusterToLBA + mov ax, bx + mov ecx, edx + mov edx, (kBoot2Segment << 4) | kBoot2Address + call readBlocks + ; TODO: error + test si, si + jnz .loop + ; fall through to boot2 +boot2: + mov dl, [BPR gDriveSelect] ; load BIOS drive number + jmp kBoot2Segment:kBoot2Address + +error: +%ifdef VERBOSE + mov di, error_str + call log_string +%endif + +hang: + hlt + jmp hang + +;-------------------------------------------------------------------------- +; ClusterToLBA - Converts cluster number to 64-bit LBA +; +; Arguments: +; EDX = cluster number +; +; Returns +; EDI:EDX = corresponding block address +; +; Assumes input cluster number is valid +; +ClusterToLBA: + push cx + xor edi, edi + sub edx, 2 + mov cl, [BPR gBlocksPerCluster] + shld edi, edx, cl + shl edx, cl + add edx, [BPR gClusterHeapOffset] + adc edi, 0 + pop cx + ret + +;-------------------------------------------------------------------------- +; BytesToBlocks - Converts byte size to blocks (rounding up to next block) +; +; Arguments: +; EBX = size in bytes +; +; Returns: +; EBX = size in blocks (rounded up) +; +; Clobbers eax, cl +; +BytesToBlocks: + xor eax, eax + inc ax + mov cl, [BPR gBytesPerBlock] + shl ax, cl + dec ax + add ebx, eax + shr ebx, cl + ret + + times (kBootBlockBytes - 2 - $ + $$) nop + dw kBootSignature +block1_end: + +;-------------------------------------------------------------------------- +; nextDirEntry - Locates the next 32-byte entry in Root Directory, +; loading block if necessary. +; +; Returns: +; CF set if end of Root Directory +; CF clear, and DS:SI points to next entry if exists +; +; Clobbers eax, ebx, ecx, edx, edi +; +nextDirEntry: + movzx ax, [BPR gsIterator + DirIterator.entry] + cmp al, [BPR gsIterator + DirIterator.entries_end] + jb .addressentry + mov ecx, [BPR gsIterator + DirIterator.lba] + mov edi, [BPR gsIterator + DirIterator.lba_high] + cmp ecx, [BPR gsIterator + DirIterator.lba_end] + jnz .readblock + mov eax, [BPR gsIterator + DirIterator.cluster] + mov edx, eax + call getRange + test ebx, ebx + jnz .nonempty + stc + ret +.nonempty: + mov [BPR gsIterator + DirIterator.cluster], eax + call ClusterToLBA + mov ecx, edx + add edx, ebx + mov [BPR gsIterator + DirIterator.lba_high], edi + mov [BPR gsIterator + DirIterator.lba_end], edx +.readblock: + mov al, 1 +%if 0 + mov edx, kRootDirBuf +%else + xor edx, edx + mov dh, kRootDirBuf >> 8 +%endif + call readLBA + ; TODO error + inc ecx + jnz .skip + inc edi + mov [BPR gsIterator + DirIterator.lba_high], edi +.skip: + mov [BPR gsIterator + DirIterator.lba], ecx + xor ax, ax +.addressentry: + mov si, ax + inc al + mov [BPR gsIterator + DirIterator.entry], al + shl si, 5 + add si, kRootDirBuf + clc + ret + +;-------------------------------------------------------------------------- +; getRange - Calculates contiguous range of clusters from FAT +; +; Arguments: +; EAX = start cluster +; +; Returns: +; EAX = next cluster after range +; EBX = number of contiguous blocks in range +; +; Range calculated is at most kMaxContigClusters clusters long +; +getRange: + push ecx + push edx + push edi + push si + xor edi, edi +%if 0 + mov edx, kFATBuf +%else + mov edx, edi + mov dh, kFATBuf >> 8 +%endif + mov ebx, edi +.loop: + cmp eax, 2 + jb .finishup + cmp eax, -9 ;kMaxCluster + jnb .finishup + cmp bx, kMaxContigClusters + jnb .finishup + inc bx + mov si, ax + and si, [BPR gsFATCache + FATCache.mask] + shl si, 2 + mov ecx, eax + inc ecx + push ecx + mov cl, [BPR gsFATCache + FATCache.shift] + shr eax, cl + cmp eax, [BPR gsFATCache + FATCache.lba] + jz .iscached + mov ecx, [BPR gFATOffset] + add ecx, eax + mov [BPR gsFATCache + FATCache.lba], eax + mov al, 1 + call readLBA + ; TODO: error? +.iscached: + pop ecx + mov eax, [kFATBuf + si] + cmp eax, ecx + jz .loop + +.finishup: + mov cl, [BPR gBlocksPerCluster] + shl ebx, cl + pop si + pop edi + pop edx + pop ecx + ret + +;-------------------------------------------------------------------------- +; readBlocks - Reads more than kMaxBlockCount blocks using LBA addressing. +; +; Arguments: +; AX = number of blocks to read (valid from 1-1280). +; EDX = pointer to where the blocks should be stored. +; EDI:ECX = block offset in partition (64 bits) +; +; Returns: +; CF = 0 success +; 1 error +; +readBlocks: + pushad + mov bx, ax + +.loop: + xor eax, eax + mov al, kMaxBlockCount + cmp bx, ax + cmovb ax, bx + call readLBA + ; TODO: error? + sub bx, ax + jz .exit + add ecx, eax + adc edi, 0 + push cx + mov cl, [BPR gBytesPerBlock] + shl eax, cl + pop cx + add edx, eax + jmp .loop + +.exit: + popad + ret + +;-------------------------------------------------------------------------- +; readLBA - Read blocks from a partition using LBA addressing. +; +; Arguments: +; AL = number of blocks to read (valid from 1-kMaxBlockCount). +; EDX = pointer to where the blocks should be stored. +; EDI:ECX = block offset in partition (64 bits) +; [gDriveSelect] = drive number (0x80 + unit number) +; [gPartitionOffset] = partition location on drive +; +; Returns: +; CF = 0 success +; 1 error +; Presently, jumps to error on BIOS-reported failure +; +readLBA: + pushad ; save all registers + push es ; save ES + mov bp, sp ; save current SP + + ; + ; Adjust to 16 bit segment:offset address + ; to allow for reading up to 64K + ; + mov bl, dl + and bx, 0xf + shr edx, 4 + mov es, dx + + ; + ; Create the Disk Address Packet structure for the + ; INT13/F42 (Extended Read Sectors) on the stack. + ; + + add ecx, [gPartitionOffset] + adc edi, [gPartitionOffset + 4] + push edi + push ecx + push es + push bx + xor ah, ah + push ax + push word 16 + + ; + ; INT13 Func 42 - Extended Read Sectors + ; + ; Arguments: + ; AH = 0x42 + ; DL = drive number (0x80 + unit number) + ; DS:SI = pointer to Disk Address Packet + ; + ; Returns: + ; AH = return status (sucess is 0) + ; carry = 0 success + ; 1 error + ; + ; Packet offset 2 indicates the number of sectors read + ; successfully. + ; + mov dl, [gDriveSelect] ; load BIOS drive number + mov si, sp + mov ah, 0x42 + int 0x13 + + jc error + + ; + ; Issue a disk reset on error. + ; Should this be changed to Func 0xD to skip the diskette controller + ; reset? + ; +; xor ax, ax ; Func 0 +; int 0x13 ; INT 13 +; stc ; set carry to indicate error + +;.exit + mov sp, bp + pop es + popad + ret + +%ifdef VERBOSE + +;-------------------------------------------------------------------------- +; Write a string with log_title_str prefix to the console. +; +; Arguments: +; DS:DI pointer to a NULL terminated string. +; +log_string: + pushad + push di + mov si, log_title_str + call print_string + pop si + call print_string + popad + ret + +;------------------------------------------------------------------------- +; Write a string to the console. +; +; Arguments: +; DS:SI pointer to a NULL terminated string. +; +; Clobber list: +; AX, BX, SI +; +print_string: + mov bx, 1 ; BH=0, BL=1 (blue) + +.loop: + lodsb ; load a byte from DS:SI into AL + test al, al ; Is it a NULL? + jz .exit ; yes, all done + mov ah, 0xE ; INT10 Func 0xE + int 0x10 ; display byte in tty mode + jmp .loop + +.exit: + ret + +%endif ; VERBOSE + +;-------------------------------------------------------------------------- +; Static data. +; + +%ifdef VERBOSE +log_title_str: db 13, 10, 'boot1x: ', 0 +init_str: db 'init', 0 +error_str: db 'error', 0 +%endif + + times (kBootBlockBytes - 4 - $ + block1_end) db 0 + dw 0, kBootSignature Index: branches/Bungo/i386/util/boot1-install/Cconfig =================================================================== --- branches/Bungo/i386/util/boot1-install/Cconfig (revision 0) +++ branches/Bungo/i386/util/boot1-install/Cconfig (revision 2840) @@ -0,0 +1,10 @@ +config BOOT1INSTALL + bool "boot1-install utility" + default y + help + Say Y here if you want to compile the boot1-install program. + boot1-install is a program 'boot1-install' that can install the stage 1 boot loader for all three file systems + boot1f32 -> FAT32 + boot1h -> HFS+ + boot1x -> exFAT + When in doubt, say "Y". Index: branches/Bungo/i386/util/boot1-install/boot1-install.c =================================================================== --- branches/Bungo/i386/util/boot1-install/boot1-install.c (revision 0) +++ branches/Bungo/i386/util/boot1-install/boot1-install.c (revision 2840) @@ -0,0 +1,852 @@ +/* + * boot1-install.c + * boot1-install + * + * Created by Zenith432 on November 19th, 2014. + * Copyright (c) 2014 Zenith432. All rights reserved. + * + * Modified December 5th, 2014 by Micky1979: Added -u option to force umount. + */ + +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#include +#include + +struct buffer_t +{ + unsigned char* _b; + size_t _s; +}; + +enum volume_kind_t +{ + _undetected = 0, + _exfat = 1, + _hfs = 2, + _msdos = 3, + _ntfs = 4, + _ext4 = 5, + _other = 255 +}; + +static int isVolumeMounted = 0; +static int isMediaWhole = 0; +static int isMediaLeaf = 0; +static enum volume_kind_t daVolumeKind = _undetected; + +static struct buffer_t bpbBlob = { NULL, 0 }; +static struct buffer_t bootBlob = { NULL, 0 }; +static struct buffer_t outputBlob = { NULL, 0 }; + +static char const UnsupportedMessage[] = "Only exFAT, FAT32 or HFS+ volumes are supported\n"; +static char const exfatID[] = "EXFAT "; +static char const fat32ID[] = "FAT32 "; +static char const devrdisk[] = "/dev/rdisk"; +static char const devdisk[] = "/dev/disk"; +static char const defaultBootFile_exfat[] = "./boot1x"; +static char const defaultBootFile_hfs[] = "./boot1h"; +static char const defaultBootFile_fat32[] = "./boot1f32"; + +static __used char const copyright[] = "Copyright 2014 Zenith432"; + +static int checkExfat(struct buffer_t const*); +static int checkFat32(struct buffer_t const*); +static int loadChunk(char const*, off_t, off_t, struct buffer_t*); +static void unsupported(void); + +#pragma mark - +#pragma mark Cleaners +#pragma mark - + +static +void free_buffer(struct buffer_t* pBuffer) +{ + assert(pBuffer); + if (pBuffer->_b) { + free(pBuffer->_b); + pBuffer->_b = NULL; + pBuffer->_s = 0; + } +} + +/* + * Uses statics + */ +static +void cleanup(void) +{ + free_buffer(&outputBlob); + free_buffer(&bootBlob); + free_buffer(&bpbBlob); +} + +#pragma mark - +#pragma mark ExFAT Processor +#pragma mark - + +static +unsigned VBRChecksum(unsigned char const* octets, size_t NumberOfBytes) +{ + unsigned Checksum = 0; + size_t Index; + for (Index = 0; Index != NumberOfBytes; ++Index) + { + if (Index == 106 || Index == 107 || Index == 112) + continue; + Checksum = ((Checksum << 31) | (Checksum >> 1)) + (unsigned) octets[Index]; + } + return Checksum; +} + +static +int calcSum(struct buffer_t const* pBootBlob, + struct buffer_t const* pBpbBlob, + struct buffer_t* pOutputBlob, + char const* pathName) +{ + unsigned char *outBuffer, *p, *q; + size_t outSize, toCopy, leftOver; + unsigned Checksum; + + assert(pBootBlob && pBpbBlob); + if (pBootBlob->_s > 9U * 512U) { + fprintf(stderr, "Boot Code must be at most 4608 bytes\n"); + return -1; + } + if (pBpbBlob->_s < 113U) { + fprintf(stderr, "BPB must be at least 113 bytes\n"); + return -1; + } + if (!checkExfat(pBpbBlob)) { + fprintf(stderr, "BPB does not contain proper exFAT signature\n"); + return -1; + } + outSize = 12U * 512U; + outBuffer = malloc(outSize); + if (!outBuffer) { + fprintf(stderr, "%s: Memory allocation failed\n", __FUNCTION__); + return -1; + } + memset(outBuffer, 0, outSize); + memcpy(outBuffer, pBootBlob->_b, pBootBlob->_s); + memcpy(&outBuffer[3], &pBpbBlob->_b[3], 8); + memset(&outBuffer[11], 0, 53); + toCopy = 120; + if (pBpbBlob->_s < toCopy) + toCopy = pBpbBlob->_s; + leftOver = 120 - toCopy; + memcpy(&outBuffer[64], &pBpbBlob->_b[64], toCopy - 64); + if (leftOver) + memset(&outBuffer[120 - leftOver], 0, leftOver); + for (toCopy = 0; toCopy != 9; ++toCopy) { + p = outBuffer + toCopy * 512U + 508U; + p[2] = 0x55U; + p[3] = 0xAAU; + if (toCopy) { + p[0] = 0U; + p[1] = 0U; + } + } + if (pathName) { + /* + * Copy OEM Parameters record + */ + struct buffer_t auxBlob = { NULL, 0 }; + if (loadChunk(pathName, 9 * 512 , 512, &auxBlob) >= 0) { + memcpy(&outBuffer[9 * 512], &auxBlob._b[0], 512); + free_buffer(&auxBlob); + } + } + Checksum = VBRChecksum(outBuffer, 11U * 512U); + p = outBuffer + 11U * 512U; + q = p + 512U; + for (; p < q; p += 4) { + *(unsigned*) p = Checksum; + } + if (pOutputBlob) { + pOutputBlob->_b = outBuffer; + pOutputBlob->_s = outSize; + } else + free(outBuffer); + return 0; +} + +#pragma mark - +#pragma mark FAT32 Processor +#pragma mark - + +static +int fat32Layout(struct buffer_t const* pBootBlob, + struct buffer_t const* pBpbBlob, + struct buffer_t* pOutputBlob) +{ + unsigned char *outBuffer; + size_t outSize; + + assert(pBootBlob && pBpbBlob); + if (pBootBlob->_s > 512U) { + fprintf(stderr, "Boot Code must be at most 512 bytes\n"); + return -1; + } + if (pBpbBlob->_s < 90U) { + fprintf(stderr, "BPB must be at least 90 bytes\n"); + return -1; + } + if (!checkFat32(pBpbBlob)) { + fprintf(stderr, "BPB does not contain proper FAT32 signature\n"); + return -1; + } + outSize = 512U; + outBuffer = malloc(outSize); + if (!outBuffer) { + fprintf(stderr, "%s: Memory allocation failed\n", __FUNCTION__); + return -1; + } + memset(outBuffer, 0, outSize); + memcpy(outBuffer, pBootBlob->_b, pBootBlob->_s); + memcpy(&outBuffer[3], &pBpbBlob->_b[3], 87); + outBuffer[510] = 0x55U; + outBuffer[511] = 0xAAU; + if (pOutputBlob) { + pOutputBlob->_b = outBuffer; + pOutputBlob->_s = outSize; + } else + free(outBuffer); + return 0; +} + +#pragma mark - +#pragma mark File Operations +#pragma mark - + +static +void writeVBR(char const* pathName, + struct buffer_t const* pBuffer, + int numCopies, + size_t expectedSize, + char const* volumeType) +{ + int fd, j; + + assert(pathName && pBuffer && volumeType); + if (pBuffer->_s != expectedSize) { + fprintf(stderr, "Unexpected %s VBR size %lu (expected %lu)\n", volumeType, pBuffer->_s, expectedSize); + return; + } + fd = open(pathName, O_WRONLY); + if (fd < 0) { + fprintf(stderr, "Unable to write boot record to %s, %s\n", pathName, strerror(errno)); + } + for (j = 0; j != numCopies; ++j) + write(fd, pBuffer->_b, pBuffer->_s); + close(fd); +} + +static +int loadChunk(char const* pathName, off_t startOffset, off_t bytesToRead, struct buffer_t* pBuffer) +{ + int fd; + ssize_t rc; + unsigned char* p; + struct stat buf; + + assert(pathName); + fd = open(pathName, O_RDONLY); + if (fd < 0) { + fprintf(stderr, "Unable to open %s, %s\n", pathName, strerror(errno)); + return -1; + } + if (bytesToRead > 0) + buf.st_size = bytesToRead; + else if (fstat(fd, &buf) < 0) { + fprintf(stderr, "Unable to fstat %s, %s\n", pathName, strerror(errno)); + close(fd); + return -1; + } + if (startOffset > 0) { + off_t t = lseek(fd, startOffset, SEEK_SET); + if (t < 0) { + fprintf(stderr, "Unable to lseek %s, %s\n", pathName, strerror(errno)); + close(fd); + return -1; + } + if (t != startOffset) { + fprintf(stderr, "lseek %s returned wrong value %lld instead of %lld\n", pathName, t, startOffset); + close(fd); + return -1; + } + if (bytesToRead <= 0) + buf.st_size -= t; + } + p = malloc((size_t) buf.st_size); + if (!p) { + fprintf(stderr, "%s: Memory allocation failed\n", __FUNCTION__); + close(fd); + return -1; + } + rc = read(fd, p, (size_t) buf.st_size); + if (rc < 0) { + fprintf(stderr, "Unable to read from %s, %s\n", pathName, strerror(errno)); + free(p); + close(fd); + return -1; + } + close(fd); + if (rc != buf.st_size) { + fprintf(stderr, "Unable to read entire chunk from %s, read %ld/%lld\n", pathName, rc, buf.st_size); + free(p); + return -1; + } + if (pBuffer) { + pBuffer->_b = p; + pBuffer->_s = (size_t) rc; + } else + free(p); + return 0; +} + +#pragma mark - +#pragma mark DiskArbitration Helpers +#pragma mark - + +static +char const* toBSDName(char const* pathName) +{ + assert(pathName); + return strncmp(pathName, &devrdisk[0], 10) ? pathName : &pathName[6]; +} + +static +char const* daReturnStr(DAReturn v) +{ + if (unix_err(err_get_code(v)) == v) + return strerror(err_get_code(v)); + switch (v) { + case kDAReturnError: + return "Error"; + case kDAReturnBusy: + return "Busy"; + case kDAReturnBadArgument: + return "Bad Argument"; + case kDAReturnExclusiveAccess: + return "Exclusive Access"; + case kDAReturnNoResources: + return "No Resources"; + case kDAReturnNotFound: + return "Not Found"; + case kDAReturnNotMounted: + return "Not Mounted"; + case kDAReturnNotPermitted: + return "Not Permitted"; + case kDAReturnNotPrivileged: + return "Not Privileged"; + case kDAReturnNotReady: + return "Not Ready"; + case kDAReturnNotWritable: + return "Not Writable"; + case kDAReturnUnsupported: + return "Unsupported"; + default: + return "Unknown"; + } +} + +static +int getDASessionAndDisk(char const* pathName, DASessionRef* pSession, DADiskRef* pDisk) +{ + DASessionRef session; + DADiskRef disk; + + assert(pathName); + session = DASessionCreate(kCFAllocatorDefault); + if (!session) { + fprintf(stderr, "DASessionCreate returned NULL\n"); + return -1; + } + disk = DADiskCreateFromBSDName(kCFAllocatorDefault, session, toBSDName(pathName)); + if (!disk) { + CFRelease(session); + fprintf(stderr, "DADiskCreateFromBSDName(%s) returned NULL\n", pathName); + return -1; + } + if (pDisk) + *pDisk = disk; + else + CFRelease(disk); + if (pSession) + *pSession = session; + else + CFRelease(session); + return 0; +} + +#pragma mark - +#pragma mark Mount/UMount +#pragma mark - + +static +void umountCallback(DADiskRef disk __unused, + DADissenterRef dissenter, + void *context) +{ + if (context && dissenter != NULL) { + *(int*) context = -1; + fprintf(stderr, "umount unsuccessful, status %s\n", daReturnStr(DADissenterGetStatus(dissenter))); + } + CFRunLoopStop(CFRunLoopGetCurrent()); +} + +static +int umount(char const* pathName, int forceUmount) +{ + DASessionRef session; + DADiskRef disk; + int rc; + + assert(pathName); + if (getDASessionAndDisk(pathName, &session, &disk) < 0) + return -1; + rc = 0; + DASessionScheduleWithRunLoop(session, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode); + DADiskUnmount(disk, forceUmount ? kDADiskUnmountOptionForce : kDADiskUnmountOptionDefault, umountCallback, &rc); + CFRunLoopRun(); + DASessionUnscheduleFromRunLoop(session, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode); + CFRelease(disk); + CFRelease(session); + return rc; +} + +static +void mountCallback(DADiskRef disk __unused, + DADissenterRef dissenter, + void *context) +{ + if (context && dissenter != NULL) { + *(int*) context = -1; + fprintf(stderr, "mount unsuccessful, status %s\n", daReturnStr(DADissenterGetStatus(dissenter))); + } + CFRunLoopStop(CFRunLoopGetCurrent()); +} + +static +int mount(char const* pathName) +{ + DASessionRef session; + DADiskRef disk; + int rc; + + assert(pathName); + if (getDASessionAndDisk(pathName, &session, &disk) < 0) + return -1; + rc = 0; + DASessionScheduleWithRunLoop(session, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode); + DADiskMount(disk, NULL, kDADiskMountOptionDefault, mountCallback, &rc); + CFRunLoopRun(); + DASessionUnscheduleFromRunLoop(session, CFRunLoopGetCurrent(), kCFRunLoopDefaultMode); + CFRelease(disk); + CFRelease(session); + return rc; +} + +#pragma mark - +#pragma mark Analyze Volume +#pragma mark - + +static +int checkExfat(struct buffer_t const* pBpbBlob) +{ + assert(pBpbBlob); + return !memcmp(&pBpbBlob->_b[3], &exfatID[0], 8); +} + +static +int checkHFS(struct buffer_t const* pBpbBlob) +{ + uint16_t sig; + + assert(pBpbBlob); + sig = OSSwapBigToHostInt16(*(uint16_t const*)&pBpbBlob->_b[0]); + return sig == 0x4244 || sig == 0x482B || sig == 0x4858; /* 'BD', 'H+', 'HX' */ +} + +static +int checkFat32(struct buffer_t const* pBpbBlob) +{ + uint16_t bytesPerSector, rootEntCnt; + uint8_t sectorsPerCluster; + + assert(pBpbBlob); + bytesPerSector = OSSwapLittleToHostInt16(*(uint16_t const*)&pBpbBlob->_b[11]); + if ((bytesPerSector & (bytesPerSector - 1U)) || + bytesPerSector < 0x200U || + bytesPerSector > 0x1000U) + return 0; + sectorsPerCluster = pBpbBlob->_b[13]; + if (!sectorsPerCluster || + (sectorsPerCluster & (sectorsPerCluster - 1U))) + return 0; + rootEntCnt = OSSwapLittleToHostInt16(*(uint16_t const*)&pBpbBlob->_b[17]); + if (rootEntCnt) + return 0; + return !memcmp(&pBpbBlob->_b[82], &fat32ID[0], 8); +} + +static +int checkSupportedVolume(enum volume_kind_t* pKind, struct buffer_t const* pBpbBlob, char const* pathName) +{ + int rc; + + assert(pKind && pBpbBlob); + rc = -1; + switch (*pKind) { + case _undetected: + if (checkExfat(pBpbBlob)) { + *pKind = _exfat; + rc = 0; + } else if (checkFat32(pBpbBlob)) { + *pKind = _msdos; + rc = 0; + } else if (pathName) { + struct buffer_t auxBlob = { NULL, 0 }; + if (loadChunk(pathName, 1024 , 512, &auxBlob) >= 0) { + if (checkHFS(&auxBlob)) { + *pKind = _hfs; + rc = 0; + } + free_buffer(&auxBlob); + } + } + break; + case _exfat: + if (checkExfat(pBpbBlob)) + rc = 0; + else + *pKind = _other; + break; + case _hfs: + if (checkHFS(pBpbBlob)) + rc = 0; + else + *pKind = _other; + break; + case _msdos: + if (checkFat32(pBpbBlob)) + rc = 0; + else + *pKind = _other; + break; + case _ntfs: + rc = 0; + break; + default: + break; + } + if (rc < 0) + unsupported(); + return rc; +} + +/* + * Uses statics + */ +static +int checkDevicePath2(char const* pathName) +{ + DASessionRef session; + DADiskRef disk; + CFDictionaryRef descDict; + CFStringRef s_ref; + CFBooleanRef b_ref; + + assert(pathName); + if (getDASessionAndDisk(pathName, &session, &disk) < 0) + return -1; + descDict = DADiskCopyDescription(disk); + if (!descDict) { + CFRelease(disk); + CFRelease(session); + fprintf(stderr, "DADiskCopyDescription(%s) returned NULL\n", pathName); + return -1; + } + if (CFDictionaryGetValueIfPresent(descDict, kDADiskDescriptionMediaWholeKey, (void const**) &b_ref) && + CFBooleanGetValue(b_ref)) + isMediaWhole = 1; + if (CFDictionaryGetValueIfPresent(descDict, kDADiskDescriptionMediaLeafKey, (void const**) &b_ref) && + CFBooleanGetValue(b_ref)) + isMediaLeaf = 1; + if (CFDictionaryContainsKey(descDict, kDADiskDescriptionVolumePathKey)) + isVolumeMounted = 1; + if (CFDictionaryGetValueIfPresent(descDict, kDADiskDescriptionVolumeKindKey, (void const**) &s_ref)) { + static char cstr_buffer[64]; + char const* cstr = CFStringGetCStringPtr(s_ref, kCFStringEncodingUTF8); + if (!cstr) { + CFStringGetCString(s_ref, &cstr_buffer[0], (CFIndex) sizeof cstr_buffer, kCFStringEncodingUTF8); + cstr = &cstr_buffer[0]; + } +#if 0 + printf("DAVolumeKind %s\n", cstr); +#endif + if (!strcmp(cstr, "exfat")) + daVolumeKind = _exfat; + else if (!strcmp(cstr, "hfs")) + daVolumeKind = _hfs; + else if (!strcmp(cstr, "msdos")) + daVolumeKind = _msdos; + else if (!strcmp(cstr, "ntfs")) + daVolumeKind = _ntfs; + else + daVolumeKind = _other; + } +#if 0 + printf(stderr, "whole %c, leaf %c, mounted %c\n", + isMediaWhole ? 'Y' : 'N', + isMediaLeaf ? 'Y' : 'N', + isVolumeMounted ? 'Y' : 'N'); +#endif +#if 0 + CFShow(descDict); +#endif + CFRelease(descDict); + CFRelease(disk); + CFRelease(session); + return 0; +} + +static +int checkDevicePath(char const* pathName) +{ + struct stat buf; + + assert(pathName); + if (strncmp(pathName, &devdisk[0], 9) != 0 && + strncmp(pathName, &devrdisk[0], 10) != 0) { + fprintf(stderr, "disk must be of form /dev/rdiskUsS or /dev/diskUsS\ndisk is %s\n", pathName); + return -1; + } + if (stat(pathName, &buf) < 0) { + fprintf(stderr, "stat on %s failed, %s\n", pathName, strerror(errno)); + return -1; + } + if (!(buf.st_mode & (S_IFCHR | S_IFBLK))) { + fprintf(stderr, "%s is not a block or character special device\n", pathName); + return -1; + } + /* + * FIXME: milk information from st_rdev - what's in it? + */ +#if 0 + printf("size of buf is %lu\n", sizeof buf); + printf("st_dev %#x\n", buf.st_dev); + printf("st_ino %llu\n", buf.st_ino); + printf("st_mode %#o\n", buf.st_mode); + printf("st_nlink %u\n", buf.st_nlink); + printf("st_uid %u\n", buf.st_uid); + printf("st_gid %u\n", buf.st_gid); + printf("st_rdev %#x\n", buf.st_rdev); + printf("st_size %llu\n", buf.st_size); + printf("st_blocks %llu\n", buf.st_blocks); + printf("st_blksize %u\n", buf.st_blksize); + printf("st_flags %#x\n", buf.st_flags); + printf("st_gen %u\n", buf.st_gen); +#endif + return 0; +} + +#pragma mark - +#pragma mark Usage +#pragma mark - + +static +void usage(char const* self) +{ + assert(self); + fprintf(stderr, "Usage: %s [-yMu] [-f boot_code_file] disk\n", self); + fprintf(stderr, " boot_code_file is an optional boot template\n"); + fprintf(stderr, " -y: don't ask any questions\n"); + fprintf(stderr, " -M: keep volume mounted while proceeding (useful for root filesystem)\n"); + fprintf(stderr, " -u: force umount (suppresses -M option if given)\n"); + fprintf(stderr, "disk is of the form /dev/rdiskUsS or /dev/diskUsS\n"); + fprintf(stderr, "default boot files are\n"); + fprintf(stderr, " boot1h for HFS+\n"); + fprintf(stderr, " boot1f32 for FAT32\n"); + fprintf(stderr, " boot1x for exFAT\n"); +} + +static +void unsupported(void) +{ + fprintf(stderr, "%s", &UnsupportedMessage[0]); +} + +#pragma mark - +#pragma mark Main +#pragma mark - + +int main(int argc, char* const argv[]) +{ + int ch; + char const* bootFile = NULL; + char const* devicePath = NULL; + int dontAsk = 0; + int keepMounted = 0; + int forceUmount = 0; + + while ((ch = getopt(argc, argv, "yMuf:")) != -1) + switch (ch) { + case 'y': + dontAsk = 1; + break; + case 'M': + keepMounted = 1; + break; + case 'u': + forceUmount = 1; + break; + case 'f': + bootFile = optarg; + break; + default: + goto usage_and_error; + } + if (optind + 1 > argc) + goto usage_and_error; + devicePath = argv[optind]; + if (geteuid() != 0) { + fprintf(stderr, "This program must be run as root\n"); + return -1; + } + if (forceUmount) + keepMounted = 0; +#if 0 + printf("bootFile %s, devicePath %s, dontAsk %d\n", bootFile, devicePath, dontAsk); +#endif + if (checkDevicePath(devicePath) < 0) + return -1; + if (checkDevicePath2(devicePath) >= 0) { + if (isMediaWhole && !isMediaLeaf) { + fprintf(stderr, "%s is a whole disk\n", devicePath); + return -1; + } + switch (daVolumeKind) { + case _undetected: + case _exfat: + case _hfs: + case _msdos: + case _ntfs: + break; + default: + unsupported(); + return -1; + } + if (isVolumeMounted && keepMounted) + isVolumeMounted = 0; + if (isVolumeMounted && umount(devicePath, forceUmount) < 0) { + if (forceUmount) + fprintf(stderr, "Unable to force umount %s, please try to umount it manually or reboot\n", devicePath); + else + fprintf(stderr, "Unable to umount %s, please 'diskutil umount' manually before running this program,\nor pass -u option to force umount it\n", devicePath); + return -1; + } + } + /* + * Note: + * Reading a non-multiple of 512 does not work on /dev/rdisk + */ + if (loadChunk(devicePath, daVolumeKind == _hfs ? 1024 : 0, 512, &bpbBlob) < 0) + goto remount_and_error; + if (checkSupportedVolume(&daVolumeKind, &bpbBlob, devicePath) < 0) + goto cleanup_and_error; + if (!bootFile) { + switch (daVolumeKind) { + case _exfat: + bootFile = &defaultBootFile_exfat[0]; + break; + case _hfs: + bootFile = &defaultBootFile_hfs[0]; + break; + case _msdos: + bootFile = &defaultBootFile_fat32[0]; + break; + default: + assert(0); + break; + } + printf("Using %s as default boot template\n", bootFile); + } + + if (loadChunk(bootFile, 0, 0, &bootBlob) < 0) + goto cleanup_and_error; + + switch (daVolumeKind) { + case _exfat: + if (calcSum(&bootBlob, &bpbBlob, &outputBlob, devicePath) < 0) + goto cleanup_and_error; + break; + case _hfs: + free_buffer(&bpbBlob); + if (bootBlob._s != 1024U) { + fprintf(stderr, "Boot Code size must be 1024 bytes\n"); + goto cleanup_and_error; + } + break; + case _msdos: + if (fat32Layout(&bootBlob, &bpbBlob, &outputBlob) < 0) + goto cleanup_and_error; + break; + default: + assert(0); + break; + } + if (!dontAsk) { + printf("About to write new boot record on %s, Are You Sure (Y/N)?", devicePath); + ch = 0; + while (ch != 'Y' && ch != 'N') + ch = getchar(); + if (ch != 'Y') { + printf("Aborted due to user request\n"); + goto cleanup_and_exit; + } + } + switch (daVolumeKind) { + case _exfat: + writeVBR(devicePath, &outputBlob, 2, 12U * 512U, "exFAT"); + break; + case _hfs: + writeVBR(devicePath, &bootBlob, 1, 1024U, "HFS+"); + break; + case _msdos: + writeVBR(devicePath, &outputBlob, 1, 512U, "FAT32"); + break; + default: + assert(0); + break; + } + +cleanup_and_exit: + cleanup(); + if (isVolumeMounted) + mount(devicePath); + return 0; + +cleanup_and_error: + cleanup(); +remount_and_error: + if (isVolumeMounted) + mount(devicePath); + return -1; + +usage_and_error: + usage(argv[0]); + return -1; +} Index: branches/Bungo/i386/util/boot1-install/Makefile =================================================================== --- branches/Bungo/i386/util/boot1-install/Makefile (revision 0) +++ branches/Bungo/i386/util/boot1-install/Makefile (revision 2840) @@ -0,0 +1,59 @@ +SRCROOT = $(abspath $(CURDIR)/../../..) +OBJROOT = $(SRCROOT)/obj/i386/util/boot1-install +SYMROOT = $(SRCROOT)/sym/i386 +DSTROOT = $(SRCROOT)/dst/i386 +DOCROOT = $(SRCROOT)/doc +IMGROOT = $(SRCROOT)/sym/cache +IMGSKELROOT = $(SRCROOT)/imgskel +CDBOOT = ${IMGROOT}/usr/standalone/i386/cdboot + +DIR = boot1-install + +include ${SRCROOT}/Make.rules + +LDFLAGS := $(LDFALGS) -mmacosx-version-min=10.5 \ +-framework CoreFoundation \ +-framework DiskArbitration \ +-Wl,-no_source_version \ +-Wl,-no_function_starts \ +-Wl,-no_data_in_code_info \ +-Wl,-no_version_load_command \ +-Wl,-no_uuid \ +-Wl,-no_dependent_dr_info + +OBJS = boot1-install.o32 \ + boot1-install.o64 + +OBJS := $(addprefix $(OBJROOT)/, $(OBJS)) + +PROGRAM = boot1-install +PROGRAM:= $(addprefix $(SYMROOT)/, $(PROGRAM)) + +ifeq ($(CONFIG_BOOT1INSTALL),y) + +all: $(SYMROOT) $(OBJROOT) $(PROGRAM) + +$(PROGRAM): $(OBJS) + @echo "\t[LD32] $(@F)_32" + @$(CC) $(CFLAGS) $(LDFLAGS) $(DEFINES) -arch i386 -o $@_32 $(filter %.o32,$^) + @echo "\t[LD64] $(@F)_64" + @$(CC) $(CFLAGS) $(LDFLAGS) $(DEFINES) -arch x86_64 -o $@_64 $(filter %.o64,$^) + @echo "\t[LIPO] $@" + @lipo -create -arch i386 $@_32 -arch x86_64 $@_64 -output $@ + @strip $@ + @rm $@_32 $@_64 + +else + +all: + +endif + + +#dependencies +-include $(OBJROOT)/Makedep + +clean-local: + @for o in $(OBJS); do if [ -f "$${o}" ];then echo " [RM] $${o}"; fi; done + @for p in $(SYMPROG); do if [ -f "$${p}" ];then echo " [RM] $${p}"; fi; done + @rm -f $(SYMPROG) $(OBJS) \ No newline at end of file Index: branches/Bungo/i386/util/sectorsize.c =================================================================== --- branches/Bungo/i386/util/sectorsize.c (revision 0) +++ branches/Bungo/i386/util/sectorsize.c (revision 2840) @@ -0,0 +1,128 @@ +// +// main.c +// sectorsize +// +// Created by Micky1979 on 15/11/14. +// Copyright (c) 2014 Insanelymac.com. All rights reserved. +// + +/* +LICENSE: + + +Creative Commons Legal Code + + +Attribution-NonCommercial 3.0 Unported + +CREATIVE COMMONS CORPORATION IS NOT A LAW FIRM AND DOES NOT PROVIDE LEGAL SERVICES. DISTRIBUTION OF THIS LICENSE DOES NOT CREATE AN ATTORNEY-CLIENT RELATIONSHIP. CREATIVE COMMONS PROVIDES THIS INFORMATION ON AN "AS-IS" BASIS. CREATIVE COMMONS MAKES NO WARRANTIES REGARDING THE INFORMATION PROVIDED, AND DISCLAIMS LIABILITY FOR DAMAGES RESULTING FROM ITS USE. + +License + +THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED. + +BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. TO THE EXTENT THIS LICENSE MAY BE CONSIDERED TO BE A CONTRACT, THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS. + +Definitions +"Adaptation" means a work based upon the Work, or upon the Work and other pre-existing works, such as a translation, adaptation, derivative work, arrangement of music or other alterations of a literary or artistic work, or phonogram or performance and includes cinematographic adaptations or any other form in which the Work may be recast, transformed, or adapted including in any form recognizably derived from the original, except that a work that constitutes a Collection will not be considered an Adaptation for the purpose of this License. For the avoidance of doubt, where the Work is a musical work, performance or phonogram, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered an Adaptation for the purpose of this License. +"Collection" means a collection of literary or artistic works, such as encyclopedias and anthologies, or performances, phonograms or broadcasts, or other works or subject matter other than works listed in Section 1(f) below, which, by reason of the selection and arrangement of their contents, constitute intellectual creations, in which the Work is included in its entirety in unmodified form along with one or more other contributions, each constituting separate and independent works in themselves, which together are assembled into a collective whole. A work that constitutes a Collection will not be considered an Adaptation (as defined above) for the purposes of this License. +"Distribute" means to make available to the public the original and copies of the Work or Adaptation, as appropriate, through sale or other transfer of ownership. +"Licensor" means the individual, individuals, entity or entities that offer(s) the Work under the terms of this License. +"Original Author" means, in the case of a literary or artistic work, the individual, individuals, entity or entities who created the Work or if no individual or entity can be identified, the publisher; and in addition (i) in the case of a performance the actors, singers, musicians, dancers, and other persons who act, sing, deliver, declaim, play in, interpret or otherwise perform literary or artistic works or expressions of folklore; (ii) in the case of a phonogram the producer being the person or legal entity who first fixes the sounds of a performance or other sounds; and, (iii) in the case of broadcasts, the organization that transmits the broadcast. +"Work" means the literary and/or artistic work offered under the terms of this License including without limitation any production in the literary, scientific and artistic domain, whatever may be the mode or form of its expression including digital form, such as a book, pamphlet and other writing; a lecture, address, sermon or other work of the same nature; a dramatic or dramatico-musical work; a choreographic work or entertainment in dumb show; a musical composition with or without words; a cinematographic work to which are assimilated works expressed by a process analogous to cinematography; a work of drawing, painting, architecture, sculpture, engraving or lithography; a photographic work to which are assimilated works expressed by a process analogous to photography; a work of applied art; an illustration, map, plan, sketch or three-dimensional work relative to geography, topography, architecture or science; a performance; a broadcast; a phonogram; a compilation of data to the extent it is protected as a copyrightable work; or a work performed by a variety or circus performer to the extent it is not otherwise considered a literary or artistic work. +"You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation. +"Publicly Perform" means to perform public recitations of the Work and to communicate to the public those public recitations, by any means or process, including by wire or wireless means or public digital performances; to make available to the public Works in such a way that members of the public may access these Works from a place and at a place individually chosen by them; to perform the Work to the public by any means or process and the communication to the public of the performances of the Work, including by public digital performance; to broadcast and rebroadcast the Work by any means including signs, sounds or images. +"Reproduce" means to make copies of the Work by any means including without limitation by sound or visual recordings and the right of fixation and reproducing fixations of the Work, including storage of a protected performance or phonogram in digital form or other electronic medium. +Fair Dealing Rights. Nothing in this License is intended to reduce, limit, or restrict any uses free from copyright or rights arising from limitations or exceptions that are provided for in connection with the copyright protection under copyright law or other applicable laws. +License Grant. Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below: +to Reproduce the Work, to incorporate the Work into one or more Collections, and to Reproduce the Work as incorporated in the Collections; +to create and Reproduce Adaptations provided that any such Adaptation, including any translation in any medium, takes reasonable steps to clearly label, demarcate or otherwise identify that changes were made to the original Work. For example, a translation could be marked "The original work was translated from English to Spanish," or a modification could indicate "The original work has been modified."; +to Distribute and Publicly Perform the Work including as incorporated in Collections; and, +to Distribute and Publicly Perform Adaptations. + + +The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats. Subject to Section 8(f), all rights not expressly granted by Licensor are hereby reserved, including but not limited to the rights set forth in Section 4(d). + +Restrictions. The license granted in Section 3 above is expressly made subject to and limited by the following restrictions: +You may Distribute or Publicly Perform the Work only under the terms of this License. You must include a copy of, or the Uniform Resource Identifier (URI) for, this License with every copy of the Work You Distribute or Publicly Perform. You may not offer or impose any terms on the Work that restrict the terms of this License or the ability of the recipient of the Work to exercise the rights granted to that recipient under the terms of the License. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties with every copy of the Work You Distribute or Publicly Perform. When You Distribute or Publicly Perform the Work, You may not impose any effective technological measures on the Work that restrict the ability of a recipient of the Work from You to exercise the rights granted to that recipient under the terms of the License. This Section 4(a) applies to the Work as incorporated in a Collection, but this does not require the Collection apart from the Work itself to be made subject to the terms of this License. If You create a Collection, upon notice from any Licensor You must, to the extent practicable, remove from the Collection any credit as required by Section 4(c), as requested. If You create an Adaptation, upon notice from any Licensor You must, to the extent practicable, remove from the Adaptation any credit as required by Section 4(c), as requested. +You may not exercise any of the rights granted to You in Section 3 above in any manner that is primarily intended for or directed toward commercial advantage or private monetary compensation. The exchange of the Work for other copyrighted works by means of digital file-sharing or otherwise shall not be considered to be intended for or directed toward commercial advantage or private monetary compensation, provided there is no payment of any monetary compensation in connection with the exchange of copyrighted works. +If You Distribute, or Publicly Perform the Work or any Adaptations or Collections, You must, unless a request has been made pursuant to Section 4(a), keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or if the Original Author and/or Licensor designate another party or parties (e.g., a sponsor institute, publishing entity, journal) for attribution ("Attribution Parties") in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; (ii) the title of the Work if supplied; (iii) to the extent reasonably practicable, the URI, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work; and, (iv) consistent with Section 3(b), in the case of an Adaptation, a credit identifying the use of the Work in the Adaptation (e.g., "French translation of the Work by Original Author," or "Screenplay based on original Work by Original Author"). The credit required by this Section 4(c) may be implemented in any reasonable manner; provided, however, that in the case of a Adaptation or Collection, at a minimum such credit will appear, if a credit for all contributing authors of the Adaptation or Collection appears, then as part of these credits and in a manner at least as prominent as the credits for the other contributing authors. For the avoidance of doubt, You may only use the credit required by this Section for the purpose of attribution in the manner set out above and, by exercising Your rights under this License, You may not implicitly or explicitly assert or imply any connection with, sponsorship or endorsement by the Original Author, Licensor and/or Attribution Parties, as appropriate, of You or Your use of the Work, without the separate, express prior written permission of the Original Author, Licensor and/or Attribution Parties. +For the avoidance of doubt: +Non-waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; +Waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License if Your exercise of such rights is for a purpose or use which is otherwise than noncommercial as permitted under Section 4(b) and otherwise waives the right to collect royalties through any statutory or compulsory licensing scheme; and, +Voluntary License Schemes. The Licensor reserves the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License that is for a purpose or use which is otherwise than noncommercial as permitted under Section 4(c). +Except as otherwise agreed in writing by the Licensor or as may be otherwise permitted by applicable law, if You Reproduce, Distribute or Publicly Perform the Work either by itself or as part of any Adaptations or Collections, You must not distort, mutilate, modify or take other derogatory action in relation to the Work which would be prejudicial to the Original Author's honor or reputation. Licensor agrees that in those jurisdictions (e.g. Japan), in which any exercise of the right granted in Section 3(b) of this License (the right to make Adaptations) would be deemed to be a distortion, mutilation, modification or other derogatory action prejudicial to the Original Author's honor and reputation, the Licensor will waive or not assert, as appropriate, this Section, to the fullest extent permitted by the applicable national law, to enable You to reasonably exercise Your right under Section 3(b) of this License (right to make Adaptations) but not otherwise. +Representations, Warranties and Disclaimer UNLESS OTHERWISE MUTUALLY AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE WORK, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU. +Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. +Termination +This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Individuals or entities who have received Adaptations or Collections from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License. +Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above. +Miscellaneous +Each time You Distribute or Publicly Perform the Work or a Collection, the Licensor offers to the recipient a license to the Work on the same terms and conditions as the license granted to You under this License. +Each time You Distribute or Publicly Perform an Adaptation, Licensor offers to the recipient a license to the original Work on the same terms and conditions as the license granted to You under this License. +If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable. +No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent. +This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You. +The rights granted under, and the subject matter referenced, in this License were drafted utilizing the terminology of the Berne Convention for the Protection of Literary and Artistic Works (as amended on September 28, 1979), the Rome Convention of 1961, the WIPO Copyright Treaty of 1996, the WIPO Performances and Phonograms Treaty of 1996 and the Universal Copyright Convention (as revised on July 24, 1971). These rights and subject matter take effect in the relevant jurisdiction in which the License terms are sought to be enforced according to the corresponding provisions of the implementation of those treaty provisions in the applicable national law. If the standard suite of rights granted under applicable copyright law includes additional rights not granted under this License, such additional rights are deemed to be included in the License; this License is not intended to restrict the license of any rights under applicable law. +Creative Commons Notice + +Creative Commons is not a party to this License, and makes no warranty whatsoever in connection with the Work. Creative Commons will not be liable to You or any party on any legal theory for any damages whatsoever, including without limitation any general, special, incidental or consequential damages arising in connection to this license. Notwithstanding the foregoing two (2) sentences, if Creative Commons has expressly identified itself as the Licensor hereunder, it shall have all rights and obligations of Licensor. + +Except for the limited purpose of indicating to the public that the Work is licensed under the CCPL, Creative Commons does not authorize the use by either party of the trademark "Creative Commons" or any related trademark or logo of Creative Commons without the prior written consent of Creative Commons. Any permitted use will be in compliance with Creative Commons' then-current trademark usage guidelines, as may be published on its website or otherwise made available upon request from time to time. For the avoidance of doubt, this trademark restriction does not form part of the License. + +Creative Commons may be contacted at http://creativecommons.org/. +*/ + +#include +#include +#include +#import +#import +#import + +void show_usage(); + +void show_usage() +{ + printf("%s", "-------------------------------------------------------------------------------\nsectorsize v1.2 Created by Micky1979 on 15/11/14\nCopyright (c) 2014 Insanelymac.com. All rights reserved.\nReleased under: Creative Commons Legal Code\nAttribution-NonCommercial 3.0 Unported\n-------------------------------------------------------------------------------\nUSAGE:\nsudo sectorsize /dev/diskX\n\nThis build is customized to be easily used with the bash/sh scripts ad return\nthe following output:\n\nPhysical XXX (if the Physical value is aqcuired..)\n\nLogical XXX (if the Physical value was not aquired for some reasons)\n\nStandard 512 (if no Logical and nor Physical value was aquired for some reasons)\n\nXXX can be 512 or 4096 (or any other aquired) as output in bytes.\n\nsectorsize exit 0 on success for the cases above. \n-------------------------------------------------------------------------------\n"); +} + +int main(int argc, const char * argv[]) { + + if (getuid()) { + show_usage(); + printf("%s", "Error: You must run this command as root user!\n-------------------------------------------------------------------------------\n"); + exit(77); + } + + if( argc > 2 ) + { + show_usage(); + printf("Error: Too many arguments\n-------------------------------------------------------------------------------\n"); + exit(64); + } + else if ( argc < 2 ) + { + show_usage(); + printf("Error: a /dev/diskX argument was expected.\n-------------------------------------------------------------------------------\n"); + exit(64); + } + + uint32_t size = 0; + uint32_t diskdevice = open(argv[1],O_RDONLY); + + if (diskdevice == -1) { + show_usage(); + printf("Error: %s not found\n-------------------------------------------------------------------------------\n", argv[1]); + exit(66); + } + + if (ioctl(diskdevice, DKIOCGETPHYSICALBLOCKSIZE, &size) != -1) { + printf("Physical %u", size); + } else if (!ioctl(diskdevice, DKIOCGETBLOCKSIZE, &size) != -1) { + printf("Locical %u\n", size); + } else { + printf("Standard 512"); + } + return 0; +} \ No newline at end of file Property changes on: branches/Bungo/i386/util/sectorsize.c ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Distribution =================================================================== --- branches/Bungo/package/Distribution (revision 0) +++ branches/Bungo/package/Distribution (revision 2840) @@ -0,0 +1,80 @@ + + + + + + + + + + + + + + Chameleon_Package_Title + + Index: branches/Bungo/package/slimpkg.sh =================================================================== --- branches/Bungo/package/slimpkg.sh (revision 0) +++ branches/Bungo/package/slimpkg.sh (revision 2840) @@ -0,0 +1,394 @@ +#!/bin/bash + +# $1 Path to store built package + +packagesidentity="org.chameleon" +packagename="Chameleon" +pkgroot="${0%/*}" +chamTemp="usr/local/chamTemp" + +COL_BLACK="\x1b[30;01m" +COL_RED="\x1b[31;01m" +COL_GREEN="\x1b[32;01m" +COL_YELLOW="\x1b[33;01m" +COL_MAGENTA="\x1b[35;01m" +COL_CYAN="\x1b[36;01m" +COL_WHITE="\x1b[37;01m" +COL_BLUE="\x1b[34;01m" +COL_RESET="\x1b[39;49;00m" + +version=$( cat version ) +stage=${version##*-} +revision=$( grep I386BOOT_CHAMELEONREVISION vers.h | awk '{ print $3 }' | tr -d '\"' ) +builddate=$( grep I386BOOT_BUILDDATE vers.h | awk '{ print $3,$4 }' | tr -d '\"' ) +timestamp=$( date -j -f "%Y-%m-%d %H:%M:%S" "${builddate}" "+%s" ) + +# ================= + +develop=$(awk "NR==6{print;exit}" ${pkgroot}/../CREDITS) +credits=$(awk "NR==10{print;exit}" ${pkgroot}/../CREDITS) +pkgdev=$(awk "NR==14{print;exit}" ${pkgroot}/../CREDITS) + +# ================= + +distributioncount=0 +xmlindent=0 + +indent[0]="\t" +indent[1]="\t\t" +indent[2]="\t\t\t" +indent[3]="\t\t\t\t" + +main () +{ + +# clean up the destination path + +rm -R -f "${1}" +echo "" +echo -e $COL_CYAN" ---------------------------------------"$COL_RESET +echo -e $COL_CYAN" Building $packagename Slim Install Package"$COL_RESET +echo -e $COL_CYAN" ---------------------------------------"$COL_RESET +echo "" + +outline[$((outlinecount++))]="${indent[$xmlindent]}" + +# build pre install package + echo "================= Preinstall =================" + ((xmlindent++)) + packagesidentity="org.chameleon" + mkdir -p ${1}/Pre/Root + mkdir -p ${1}/Pre/Scripts + ditto --noextattr --noqtn ${1%/*/*}/revision ${1}/Pre/Scripts/Resources/revision + ditto --noextattr --noqtn ${1%/*/*}/version ${1}/Pre/Scripts/Resources/version + cp -f ${pkgroot}/Scripts/Main/preinstall ${1}/Pre/Scripts + echo " [BUILD] Pre " + buildpackage "${1}/Pre" "/" "" "start_visible=\"false\" start_selected=\"true\"" >/dev/null 2>&1 +# End build pre install package + +# build core package + echo "================= Core =================" + packagesidentity="org.chameleon" + mkdir -p ${1}/Core/Root/usr/local/bin + mkdir -p ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot0 ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot0md ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot1f32 ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot1h ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot1he ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/boot1hp ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/cdboot ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/chain0 ${1}/Core/Root/usr/standalone/i386 + ditto --noextattr --noqtn ${1%/*}/i386/fdisk440 ${1}/Core/Root/usr/local/bin + ditto --noextattr --noqtn ${1%/*}/i386/bdmesg ${1}/Core/Root/usr/local/bin + local coresize=$( du -hkc "${1}/Core/Root" | tail -n1 | awk {'print $1'} ) + echo " [BUILD] i386 " + buildpackage "${1}/Core" "/" "0" "start_visible=\"false\" start_selected=\"true\"" >/dev/null 2>&1 +# End build core package + +# build install type + echo "================= Chameleon =================" + outline[$((outlinecount++))]="${indent[$xmlindent]}" + choices[$((choicescount++))]="\t\n\t\n" + ((xmlindent++)) + packagesidentity="org.chameleon.type" + + # build new install package + mkdir -p ${1}/New/Root + echo "" > "${1}/New/Root/install_type_new" + echo " [BUILD] New " + buildpackage "${1}/New" "/$chamTemp" "" "start_enabled=\"true\" selected=\"exclusive(choices['Upgrade'])\"" >/dev/null 2>&1 + # End build new install package + + # build upgrade package + mkdir -p ${1}/Upgrade/Root + echo "" > "${1}/Upgrade/Root/install_type_upgrade" + echo " [BUILD] Upgrade " + buildpackage "${1}/Upgrade" "/$chamTemp" "" "start_selected=\"false\" selected=\"exclusive(choices['New'])\"" >/dev/null 2>&1 + # End build upgrade package + + ((xmlindent--)) + outline[$((outlinecount++))]="${indent[$xmlindent]}" +# End build install type + +# build Chameleon package + echo "================= Chameleon =================" + outline[$((outlinecount++))]="${indent[$xmlindent]}" + choices[$((choicescount++))]="\t\n\t\n" + ((xmlindent++)) + + # build standard package + mkdir -p ${1}/Standard/Root + mkdir -p ${1}/Standard/Scripts/Resources + cp -f ${pkgroot}/Scripts/Main/Standardpostinstall ${1}/Standard/Scripts/postinstall + ditto --arch i386 `which SetFile` ${1}/Standard/Scripts/Resources/SetFile + ditto --noextattr --noqtn ${1%/*/*}/revision ${1}/Standard/Scripts/Resources/revision + ditto --noextattr --noqtn ${1%/*/*}/version ${1}/Standard/Scripts/Resources/version + echo " [BUILD] Standard " + buildpackage "${1}/Standard" "/" "${coresize}" "start_enabled=\"true\" selected=\"exclusive(choices['EFI']) && exclusive(choices['noboot'])\"" >/dev/null 2>&1 + # End build standard package + + # build efi package + mkdir -p ${1}/EFI/Root + mkdir -p ${1}/EFI/Scripts/Resources + cp -f ${pkgroot}/Scripts/Main/ESPpostinstall ${1}/EFI/Scripts/postinstall + ditto --arch i386 `which SetFile` ${1}/EFI/Scripts/Resources/SetFile + ditto --noextattr --noqtn ${1%/*/*}/revision ${1}/EFI/Scripts/Resources/revision + ditto --noextattr --noqtn ${1%/*/*}/version ${1}/EFI/Scripts/Resources/version + echo " [BUILD] EFI " + buildpackage "${1}/EFI" "/" "${coresize}" "start_visible=\"systemHasGPT()\" selected=\"exclusive(choices['Standard']) && exclusive(choices['noboot'])\"" >/dev/null 2>&1 + # End build efi package + + # build reset choice package + mkdir -p ${1}/noboot/Root + echo " [BUILD] Reset choice " + buildpackage "${1}/noboot" "/$chamTemp" "" "selected=\"exclusive(choices['Standard']) && exclusive(choices['EFI'])\"" >/dev/null 2>&1 + # End build reset choice package + + ((xmlindent--)) + outline[$((outlinecount++))]="${indent[$xmlindent]}" +# End build Chameleon package + +# build Modules package + echo "================= Modules =================" + ############################### + # Supported Modules # + ############################### + # klibc.dylib # + # Resolution.dylib # + # uClibcxx.dylib # + # Keylayout.dylib # + ############################### + if [ "$(ls -A "${1%/*}/i386/modules")" ]; then + { + outline[$((outlinecount++))]="${indent[$xmlindent]}" + choices[$((choicescount++))]="\t\n\t\n" + ((xmlindent++)) + packagesidentity="org.chameleon.modules" +# - + if [ -e ${1%/*}/i386/modules/klibc.dylib ]; then + { + mkdir -p ${1}/klibc/Root + ditto --noextattr --noqtn ${1%/*}/i386/modules/klibc.dylib ${1}/klibc/Root + echo " [BUILD] klibc " + buildpackage "${1}/klibc" "/$chamTemp/Extra/modules" "" "start_selected=\"false\"" >/dev/null 2>&1 + } + fi +# - + if [ -e ${1%/*}/i386/modules/uClibcxx.dylib ]; then + { + mkdir -p ${1}/uClibc/Root + ditto --noextattr --noqtn ${1%/*}/i386/modules/uClibcxx.dylib ${1}/uClibc/Root + ditto --noextattr --noqtn ${1%/*}/i386/modules/klibc.dylib ${1}/uClibc/Root + echo " [BUILD] uClibc++ " + buildpackage "${1}/uClibc" "/$chamTemp/Extra/modules" "" "start_selected=\"false\"" >/dev/null 2>&1 + } + fi +# - + if [ -e ${1%/*}/i386/modules/Resolution.dylib ]; then + { + mkdir -p ${1}/AutoReso/Root + ditto --noextattr --noqtn ${1%/*}/i386/modules/Resolution.dylib ${1}/AutoReso/Root + echo " [BUILD] Resolution " + buildpackage "${1}/AutoReso" "/$chamTemp/Extra/modules" "" "start_selected=\"false\"" >/dev/null 2>&1 + } + fi +# - + if [ -e ${1%/*}/i386/modules/Keylayout.dylib ]; then + { + mkdir -p ${1}/Keylayout/Root + ditto --noextattr --noqtn ${1%/*}/i386/modules/Keylayout.dylib ${1}/Keylayout/Root + echo " [BUILD] Keylayout " + buildpackage "${1}/Keylayout" "/$chamTemp/Extra/modules" "" "start_selected=\"false\"" >/dev/null 2>&1 + } + fi + + ((xmlindent--)) + outline[$((outlinecount++))]="${indent[$xmlindent]}" + } + else + { + echo " -= no modules to include =-" + } + fi +# End build Modules packages + +# build post install package + echo "================= Post =================" + packagesidentity="org.chameleon" + mkdir -p ${1}/Post/Root + mkdir -p ${1}/Post/Scripts + cp -f ${pkgroot}/Scripts/Main/postinstall ${1}/Post/Scripts + ditto --noextattr --noqtn ${1%/*/*}/revision ${1}/Post/Scripts/Resources/revision + ditto --noextattr --noqtn ${1%/*/*}/version ${1}/Post/Scripts/Resources/version + echo " [BUILD] Post " + buildpackage "${1}/Post" "/" "" "start_visible=\"false\" start_selected=\"true\"" >/dev/null 2>&1 +# End build post install package + +#((xmlindent--)) +outline[$((outlinecount++))]="${indent[$xmlindent]}" + +# build meta package + + makedistribution "${1}" "${2}" "${3}" "${4}" #"${5}" + +# clean up + + rm -R -f "${1}" + +} + +fixperms () +{ + # $1 path + find "${1}" -type f -exec chmod 644 {} \; + find "${1}" -type d -exec chmod 755 {} \; + chown -R 0:0 "${1}" +} + +buildpackage () +{ +# $1 Path to package to build containing Root and or Scripts +# $2 Install Location +# $3 Size +# $4 Options + +if [ -d "${1}/Root" ] && [ "${1}/Scripts" ]; then + + local packagename="${1##*/}" + local identifier=$( echo ${packagesidentity}.${packagename//_/.} | tr [:upper:] [:lower:] ) + find "${1}" -name '.DS_Store' -delete + local filecount=$( find "${1}/Root" | wc -l ) + if [ "${3}" ]; then + local installedsize="${3}" + else + local installedsize=$( du -hkc "${1}/Root" | tail -n1 | awk {'print $1'} ) + fi + local header="\n\n" + header+="\t\n" + rm -R -f "${1}/Temp" + + [ -d "${1}/Temp" ] || mkdir -m 777 "${1}/Temp" + [ -d "${1}/Root" ] && mkbom "${1}/Root" "${1}/Temp/Bom" + + if [ -d "${1}/Scripts" ]; then + header+="\t\n" + for script in $( find "${1}/Scripts" -type f \( -name 'pre*' -or -name 'post*' \) ) + do + header+="\t\t<${script##*/} file=\"./${script##*/}\"/>\n" + done + header+="\t\n" + chown -R 0:0 "${1}/Scripts" + pushd "${1}/Scripts" >/dev/null + find . -print | cpio -o -z -H cpio > "../Temp/Scripts" + popd >/dev/null + fi + + header+="" + echo -e "${header}" > "${1}/Temp/PackageInfo" + pushd "${1}/Root" >/dev/null + find . -print | cpio -o -z -H cpio > "../Temp/Payload" + popd >/dev/null + pushd "${1}/Temp" >/dev/null + + xar -c -f "${1%/*}/${packagename// /}.pkg" --compression none . + + popd >/dev/null + + outline[$((outlinecount++))]="${indent[$xmlindent]}" + + if [ "${4}" ]; then + local choiceoptions="\t\t${4}" + fi + choices[$((choicescount++))]="\t\n\t\t#${packagename// /}.pkg\n\t\n" + rm -R -f "${1}" +fi +} + +makedistribution () +{ + rm -f "${1%/*}/${packagename// /}"*.pkg + + find "${1}" -type f -name '*.pkg' -depth 1 | while read component + do + mkdir -p "${1}/${packagename}/${component##*/}" + pushd "${1}/${packagename}/${component##*/}" >/dev/null + xar -x -f "${1%}/${component##*/}" + popd >/dev/null + done + + ditto --noextattr --noqtn "${pkgroot}/Distribution" "${1}/${packagename}/Distribution" + ditto --noextattr --noqtn "${pkgroot}/Resources" "${1}/${packagename}/Resources" + + find "${1}/${packagename}/Resources" -type d -name '.svn' -exec rm -R -f {} \; 2>/dev/null + + for (( i=0; i < ${#outline[*]} ; i++)); + do + echo -e "${outline[$i]}" >> "${1}/${packagename}/Distribution" + done + + for (( i=0; i < ${#choices[*]} ; i++)); + do + echo -e "${choices[$i]}" >> "${1}/${packagename}/Distribution" + done + + echo "" >> "${1}/${packagename}/Distribution" + + perl -i -p -e "s/%CHAMELEONVERSION%/${version%%-*}/g" `find "${1}/${packagename}/Resources" -type f` + perl -i -p -e "s/%CHAMELEONREVISION%/${revision}/g" `find "${1}/${packagename}/Resources" -type f` + +# Adding Developer and credits + perl -i -p -e "s/%DEVELOP%/${develop}/g" `find "${1}/${packagename}/Resources" -type f` + perl -i -p -e "s/%CREDITS%/${credits}/g" `find "${1}/${packagename}/Resources" -type f` + perl -i -p -e "s/%PKGDEV%/${pkgdev}/g" `find "${1}/${packagename}/Resources" -type f` + + stage=${stage/RC/Release Candidate } + stage=${stage/FINAL/2.0 Final} + perl -i -p -e "s/%CHAMELEONSTAGE%/${stage}/g" `find "${1}/${packagename}/Resources" -type f` + + find "${1}/${packagename}" -name '.DS_Store' -delete + pushd "${1}/${packagename}" >/dev/null + xar -c -f "${1%/*}/${packagename// /}-${version}-r${revision}.pkg" --compression none . + popd >/dev/null + +# Here is the place for assign a Icon to the pkg +# command use to generate the file: +# ditto -c -k --sequesterRsrc --keepParent Icon.icns Icon.zip +# ---- + ditto -xk "${pkgroot}/Icons/pkg.zip" "${pkgroot}/Icons/" + DeRez -only icns "${pkgroot}/Icons/Icons/pkg.icns" > tempicns.rsrc + Rez -append tempicns.rsrc -o "${1%/*}/$packagename-${version}-r$revision.pkg" + SetFile -a C "${1%/*}/$packagename-${version}-r$revision.pkg" + rm -f tempicns.rsrc + rm -rf "${pkgroot}/Icons/Icons" +# End + + echo "" + + echo -e $COL_GREEN" --------------------------"$COL_RESET + echo -e $COL_GREEN" Building process complete!"$COL_RESET + echo -e $COL_GREEN" --------------------------"$COL_RESET + echo "" + echo -e $COL_GREEN" Build info." + echo -e $COL_GREEN" ===========" + echo -e $COL_BLUE" Package name: "$COL_RESET"$packagename-${version}-r$revision.pkg" + echo -e $COL_BLUE" MD5: "$COL_RESET"$md5" + echo -e $COL_BLUE" Version: "$COL_RESET"$version" + echo -e $COL_BLUE" Stage: "$COL_RESET"$stage" + echo -e $COL_BLUE" Date/Time: "$COL_RESET"$builddate" + echo "" + +} + +main "${1}" "${2}" "${3}" "${4}" #"${5}" + Property changes on: branches/Bungo/package/slimpkg.sh ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts.templates/AddOption/postinstall =================================================================== --- branches/Bungo/package/Scripts.templates/AddOption/postinstall (revision 0) +++ branches/Bungo/package/Scripts.templates/AddOption/postinstall (revision 2840) @@ -0,0 +1,64 @@ +#!/bin/bash + +set -u + +configFile='/private/tmp/InstallConfig.plist' +v_mntptDev=$( /usr/libexec/plistbuddy -c "Print :ramdisk" ${configFile} | sed -e 's/[[:blank:]]*//g' ) +v_mntpt=$( LC_ALL=C diskutil info ${v_mntptDev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//' ) + +key="@optionKey@" +value="@optionValue@" +type="@optionType@" +logName="@LOG_FILENAME@" + +# Check if target volume exists +if [[ ! -d "$v_mntpt" ]]; then + echo "$v_mntpt volume does not exist!" >&2 + exit 1 +fi + +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" + +exec > >(tee -a "${v_mntpt}/${logName}") 2>&1 + +echo "$mainLine" +echo "Writing boot option: ${key}=${value}" + +key="${key// /\\ }" # Escape spaces +value="${value// /\\ }" # Escape spaces + +bootPListFile="${v_mntpt}/Extra/org.chameleon.Boot.plist" + +case "$type" in + bool|text) + /usr/libexec/plistbuddy -c "Add :${key} string ${value}" "$bootPListFile" + ;; + list) + current_values=$( /usr/libexec/plistbuddy -c "Print :${key}" \ + "$bootPListFile" 2>/dev/null ) + result=$? + current_values="${current_values// /\\ }" # Escape spaces + current_values="${current_values//\"/\\\"}" # Escape double quotes + + if [[ $result -eq 0 ]]; then + # Append our new values + if [[ "$current_values" = "" ]]; then + new_values="${value}" + else + new_values="${current_values}\ ${value}" + fi + /usr/libexec/plistbuddy -c "Set :${key} ${new_values}" \ + "$bootPListFile" + else + # Create a new option + new_values="${value}" + /usr/libexec/plistbuddy -c "Add :${key} string ${new_values}" \ + "$bootPListFile" + fi + ;; +esac + +echo "$subLine" +exit 0 Property changes on: branches/Bungo/package/Scripts.templates/AddOption/postinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts.templates/Post/postinstall =================================================================== --- branches/Bungo/package/Scripts.templates/Post/postinstall (revision 0) +++ branches/Bungo/package/Scripts.templates/Post/postinstall (revision 2840) @@ -0,0 +1,97 @@ +#!/bin/bash + +# $1 = Full path to the installation package the installer app is processing +# $2 = Full path to the installation destination +# $3 = Installation volume (mountpoint) to receive the payload +# $4 = Root directory for the system + +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" + +if [ "$3" == "/" ] +then + targetVolume="/Volumes/"$( ls -1F /Volumes | sed -n 's:@$::p' ) + # really need to do that?? Unix "component" path assume are between "/" + # ie current volume can be also /// or ///Volumes///MyVolume w/o problem! +else + targetVolume="$3" +fi + +configFile='/private/tmp/InstallConfig.plist' +v_mntptDev=$( /usr/libexec/plistbuddy -c "Print :ramdisk" ${configFile} ) +v_mntpt=$( LC_ALL=C diskutil info ${v_mntptDev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//' ) + +targetDevice=$( /usr/libexec/plistbuddy -c "Print :targetdev" ${configFile} ) +choicedVolume=$( LC_ALL=C diskutil info ${targetDevice} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//' ) +backupRootDir="${targetVolume}/Chameleon.Backups" +backupDir=$( /usr/libexec/plistbuddy -c "Print :backupDir" ${configFile} ) + +# Check target exists +if [ ! -d "${targetVolume}" ]; then + echo "Target volume does not exist !" >&2 + exit 1 +fi +if [ ! -d "${v_mntpt}" ]; then + echo "Ram disk volume does not exist !" >&2 + exit 1 +fi +if [ ! -d "${choicedVolume}" ]; then + echo "${choicedVolume} volume does not exist !" >&2 + exit 1 +fi + +logName="@LOG_FILENAME@" + +exec > >(tee -a "${v_mntpt}/${logName}") 2>&1 + +echo "$mainLine" +echo "Running Post postinstall script" +echo "Target volume = ${choicedVolume}" +echo "$subLine" + +# Replace/Copying the Extra folder +echo "Moving Extra folder to ${choicedVolume}" +cp -R "${v_mntpt}/Extra" "${choicedVolume}"/ + +echo "NOTE: any Themes or modules you have must be there since this now is the boot partition," +echo " ACPI tables, SMBios.plist and the org.chameleon.Boot.plist (with custom settings" +echo " for the target OSX must be in each partition that contanin it.)" + +echo "$subLine" +echo "Post postinstall script complete" +echo "$mainLine" + +# Update Rights +chmod 777 "${choicedVolume}"/Extra 2>/dev/null +chmod 666 "${choicedVolume}"/Extra/*.plist 2>/dev/null +chmod 666 "${choicedVolume}/Extra/${logName}" 2>/dev/null +# if an Extra/Extensions exist... we can repair the permission??? + +# Check Backup folder (is always on the target Volume) + +if [ -d "${backupRootDir}/${backupDir}" ]; then + # Remove empty directories + find "${backupRootDir}" -type d -depth -empty -exec rmdir {} \; +fi + +# copying the installer log inside the Extra folder +if [[ $( /usr/libexec/plistbuddy -c "Print bootloader" ${configFile} ) == "true" ]];then + # if we have installed the bootloader, this is a new log + cat "${v_mntpt}/${logName}" > "${choicedVolume}/Extra/${logName}" +else + # ..otherwise adding the new log to the existing one (if exist) + cat "${v_mntpt}/${logName}" >> "${choicedVolume}/Extra/${logName}" +fi + + +# Umount the Ram Disk & cleaning +rm -f /private/tmp/InstallConfig.plist +rm -f "${targetVolume}/EXTRAROOTDIR" +exec 1<&- # Restore stdout and close file descriptor #1 before umount the Ram Disk +umount -f $v_mntptDev > /dev/null +hdiutil detach $v_mntptDev + + + Property changes on: branches/Bungo/package/Scripts.templates/Post/postinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts.templates/Pre/preinstall =================================================================== --- branches/Bungo/package/Scripts.templates/Pre/preinstall (revision 0) +++ branches/Bungo/package/Scripts.templates/Pre/preinstall (revision 2840) @@ -0,0 +1,91 @@ +#!/bin/bash + +# Creates text file named '@LOG_FILENAME@' +# at the root of the target volume. This is to give the user +# a record of the installation process and also to show why +# possibly the installation process failed (even though the +# package installer ends reading 'Installation Successful'). + +# This script also prepares the Ram Disk and the InstallConfig.plist +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" + +if [ "$3" == "/" ] +then + targetVolume="/Volumes/"$( ls -1F /Volumes | sed -n 's:@$::p' ) +else + targetVolume="$3" +fi + +v_mntpt="/Volumes/BOOTRAMDISK" +size='409600' # is the size of a standard EFI partition +logName="@LOG_FILENAME@" +configFile="/private/tmp/InstallConfig.plist" + +# Functions to create/umount the RAM DISK +UMOUNT_VDISK() { + umount -f "${1}" + hdiutil detach "${1}" +} + +RAM_DISK() { + if [ $( df | awk '{$1=$2=$3=$4=$5=$6=$7=$8=""; print $0}' | sed -e 's/^[ \t]*//' | \ + grep -x "${v_mntpt}" ) ]; then + devToUmount=$( LC_ALL=C diskutil info "${v_mntpt}" | grep -i 'Device Node:' | awk '{print $3}' ) + UMOUNT_VDISK $devToUmount + fi + echo "CREATING RAM DISK" +# mkdir -p $v_mntpt + + dev=$(hdiutil attach -nomount ram://${size}) + if [ $? -eq 0 ] ; then + diskutil erasevolume FAT32 "BOOTRAMDISK" ${dev} + else + echo "Failed to create the Ram Disk, installation aborted!" + exit 1 + fi + + rm -f $configFile + # adding the Ram disk device to InstallConfig.plist to be shared with other packages + /usr/libexec/PlistBuddy -c "Add :ramdisk string ${dev}" $configFile +} + +RAM_DISK + +# ensure that ram disk has "/Volumes/BOOTRAMDISK" mount point +v_mntpt=$( LC_ALL=C diskutil info ${dev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//' ) +if [ ! -d "${v_mntpt}" ]; then + echo "Ram Disk not found!" + exit +fi + +touch "${v_mntpt}/${logName}" +exec > >(tee -a "${v_mntpt}/${logName}") 2>&1 + +echo "$mainLine" +echo "Pre-Install Script" +echo "$subLine" + +# creating a symlink (from Clover) +echo "Creating ${targetVolume}/EXTRAROOTDIR symlink targeting ${v_mntpt}" +ln -sf "${v_mntpt}" "${targetVolume}/EXTRAROOTDIR" + +# Preparing Backing up of Chameleon files +backupRootDir="${targetVolume}/Chameleon.Backups" +backupDir=$( date -j "+%F-%Hh%M" ) + +# Create the backup directory +mkdir -p "${backupRootDir}/${backupDir}" + +# Remove the invisible flag of files in the backups +chflags -R nohidden "${backupRootDir}" + +# adding the backupdir name to InstallConfig.plist to be shared with other packages +/usr/libexec/PlistBuddy -c "Add :backupDir string ${backupDir}" $configFile >/dev/null + +diskutil list >> "${v_mntpt}/${logName}" +echo "$subLine" +echo "END - Pre-Install Script" + +exit 0 Property changes on: branches/Bungo/package/Scripts.templates/Pre/preinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts.templates/InstallTheme/postinstall =================================================================== --- branches/Bungo/package/Scripts.templates/InstallTheme/postinstall (revision 0) +++ branches/Bungo/package/Scripts.templates/InstallTheme/postinstall (revision 2840) @@ -0,0 +1,36 @@ +#!/bin/bash + +set -u + +configFile='/private/tmp/InstallConfig.plist' +v_mntptDev=$( /usr/libexec/plistbuddy -c "Print :ramdisk" ${configFile} | sed -e 's/[[:blank:]]*//g' ) +v_mntpt=$( LC_ALL=C diskutil info ${v_mntptDev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//' ) + +themeName="@themeName@" +themeDir="@themeDir@" +logName="@LOG_FILENAME@" + +# Check if target volume exists +if [[ ! -d "${v_mntpt}" ]]; then + echo "$v_mntpt volume does not exist!" >&2 + exit 1 +fi + +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" + +exec > >(tee -a "${v_mntpt}/${logName}") 2>&1 + +echo "$mainLine" +echo "Installing Theme $themeName" +echo "$subLine" + +if [[ -d "${v_mntpt}/Extra/Themes/$themeDir" ]]; then + echo "Theme $themeName installed" +else + echo "*** ERROR:$themeName not installed" +fi + +echo "$subLine" +exit 0 Property changes on: branches/Bungo/package/Scripts.templates/InstallTheme/postinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts.templates/InstallModule/postinstall =================================================================== --- branches/Bungo/package/Scripts.templates/InstallModule/postinstall (revision 0) +++ branches/Bungo/package/Scripts.templates/InstallModule/postinstall (revision 2840) @@ -0,0 +1,37 @@ +#!/bin/bash + +set -u + +configFile='/private/tmp/InstallConfig.plist' +v_mntptDev=$( /usr/libexec/plistbuddy -c "Print :ramdisk" ${configFile} | sed -e 's/[[:blank:]]*//g' ) +v_mntpt=$( LC_ALL=C diskutil info ${v_mntptDev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//' ) + +moduleName="@moduleName@" +moduleFile="@moduleFile@" +logName="@LOG_FILENAME@" + +# Check if target volume exists +if [[ ! -d "${v_mntpt}" ]]; then + echo "Ram Disk not found volume does not exist!" >&2 +# exit 1 +fi + +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" + +exec > >(tee -a "${v_mntpt}/${logName}") 2>&1 + +echo "$mainLine" +echo "Installing $moduleFile" +echo "$subLine" + +if [[ -f "${v_mntpt}/Extra/modules/$moduleFile" ]]; then + echo "$moduleName installed to Ram disk" +else + echo "*** ERROR:$moduleName not installed" +fi + +echo "$subLine" + +exit 0 Property changes on: branches/Bungo/package/Scripts.templates/InstallModule/postinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts.templates/CleanOptions/clean_bootplist.pl =================================================================== --- branches/Bungo/package/Scripts.templates/CleanOptions/clean_bootplist.pl (revision 0) +++ branches/Bungo/package/Scripts.templates/CleanOptions/clean_bootplist.pl (revision 2840) @@ -0,0 +1,95 @@ +#!/usr/bin/perl + +use strict; +use YAML::Syck; + +our $target_volume; +our $boot_plist_filepath; + +our $yaml_file="@YAML_FILE@"; + +if ($#ARGV < 0) { + print stderr "A target volume is needed\n"; +} else { + $target_volume=$ARGV[0]; +} + +$boot_plist_filepath = "${target_volume}/Extra/org.chameleon.Boot.plist"; +if ( -f "$boot_plist_filepath" ) { + main("$yaml_file"); +} + +sub _do_cmd { + my ($cmd, $key, $value) = @_; + my $out; + + $key =~ s/([\s])/\\$1/g; # Escape characters in key + $value =~ s/([\s"])/\\$1/g; # Escape characters in value (space & ") + my $plistbuddy_command="$cmd :$key $value"; + + open ( OUTPUT, "-|", '/usr/libexec/plistbuddy', "-c", "$plistbuddy_command", "$boot_plist_filepath" ); + my $exit_code = $?; + chomp($out = ); + close OUTPUT; + return $out; +} + +sub get_boot_plist_option { + my ($option) = @_; + return _do_cmd( "Print", "$option"); +} + +sub delete_boot_plist_option { + my ($option) = @_; + return _do_cmd( "Delete", "$option"); +} + +sub delete_boot_plist_text_option { + my ($option, $values) = @_; + my $current_value = get_boot_plist_option "$option"; + if ( $current_value ne "") { + foreach my $recognized_value (@{$values}) { + if ($recognized_value eq $current_value) { + return _do_cmd( "Delete", "$option"); + } + } + } + return ""; +} + +sub delete_boot_plist_list_option { + my ($option, $values) = @_; + my $current_value = get_boot_plist_option "$option"; + if ( $current_value ne "") { + my %count; + my @new_list; + foreach my $value (@{$values}) { + $count{$value} = 1; + } + foreach my $value (split(/\s+/,$current_value)) { + if ( not exists $count{$value} ) { + push @new_list, $value; + } + } + return _do_cmd( "Set", $option, join(' ',@new_list) ); + } + return ""; +} + +sub main() { + # Remove all options that installer can managed + my ($yaml_file) = @_; + my $hash_ref = LoadFile($yaml_file) or die "Can't open yaml file\n"; + foreach my $option ( keys %{$hash_ref} ) { + my $type = $hash_ref->{$option}->{type}; + if ( $type =~ /^bool$/i ) { + delete_boot_plist_option($option); + } elsif ( $type =~ /^text$/i ) { + delete_boot_plist_text_option( $option, + $hash_ref->{$option}->{values} ); + } elsif ( $type =~ /^list$/i ) { + delete_boot_plist_list_option( $option, + $hash_ref->{$option}->{values} ); + } + } +} Property changes on: branches/Bungo/package/Scripts.templates/CleanOptions/clean_bootplist.pl ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Changes.txt =================================================================== --- branches/Bungo/package/Changes.txt (revision 0) +++ branches/Bungo/package/Changes.txt (revision 2840) @@ -0,0 +1,85 @@ +- Micky1979 - Add sectorsize is a tiny command line made to detect the Physical and logical sector size of your hard disk. + +- JrCs - boot1-install can show ntfs volume. + +- Zenith432 - Add boot1-install utility that can install the stage 1 boot loader. + +- ErmaC - Auto increase year for pkg resource. + +- JrCs - Add translation to the project. + +- ErmaC - Update Language for whobuild resources. + +- JrCs - Improved Installer - Start using template scripts + +- JrCs - Look for a com.apple.Boot.plist file for users upgrading from legacy installations + +- Only create Chameleon.Backups and write to the install log, if required. + +- JrCs - Improved backup of chameleon files + +- JrCs - Automaticaly select previous options + +- JrCs - Improved buildpkg script - Simplify the construction of options menu - Add a lot of checks to avoid the construction of a bad XML file + +- Change Localizable.strings to match the recent changes to the install process. + +- Improve messages written to the installation log. + +- Change upgrade install to only backup /Extra folder if necessary. + +- JrCs - Enhanced installer - We can add an option without an associated package - We can have an option that install +multiple packages - No more "fake" option (none) - The package that is build is already compressed (no need to +compress it again) + +- JrCs - Update postinstall script - Fix a bug where kernel flags are not set properly - Fix indent - Add some comments. + +- JrCs - Automaticaly select upgrade option if /Extra/org.chameleon.Boot.plist file exists on the target volume. + +- JrCs - Keylayout / Keymaps fixed and made a mandatory install when a keymap is chosen. + +- Add option to upgrade an existing install - or more exactly, merge newly selected options with an existing /Extra folder. Also, rename an existing /Extra/com.apple.Boot.plist to /Extra/org.chameleon.Boot.plist. + +- Change the layout/wording displayed in the installer (English only) and give it a facelift. Adjust background image to scale to fit in the window. + +- Set exclusive 'None' option to default choice. + +- Add code to detect and make necessary changes to avoid possibility of a b1f:error or boot1:error should the user decide to install a secondary boot partition. + +- Fix bug installing to /Volumes/EFI + +- Add crazybirdy's Chinese translation to zh-CN and zh-TW resources. + +- Add dmaazar's boot0workV2 changes to ../i386/boot0/boot0md.s. + +- Added check for an existing Chameleon installation on a different partition of same target disk to help stop new +users from confusing themselves. If found, the install process will exit and write the reason to the install log. + +- Re-organised Scripts folder in to Main and Sub scripts. The main scripts for the standard and EFI system partition +installation options have been changed to used the code that I'd previously re-structured. The Sub scripts are the +original chameleon installer scripts with maybe some tweaks/additions, split in to separate scripts. These scripts +include the previous additions I'd made including checking for FAT16 partitions, installing boot0md or boot0 depending +on whether or not a Windows installation is found and writing boot1h or boot1f32 depending on installing to HFS or +FAT32 format partitions. + +- All references of Boot0hfs removed and now replaced with Boot0md. + +- Installer log added to now dump a log of useful info about the install process to the root of the selected target. + +- Boot options re-thought and are now automatically created at compile time from simple lists, rather than manage a +separate file of code of each option. + +- Added missing useKernelCache boot option. + +- Easily enable a list to be exclusive or not. + +- The main Post script now creates the org.chameleon.Boot.plist based on the modules / options / key layouts / themes +selected by the user. + +- Only create and /Extra folder if there's something to go in it. + +- Backup an existing /Extra folder to /Extra-OLD(date & time). + +- slimpkg.sh changed to match the revised code in buildpkg.sh where applicable. + +- English Localizable.strings revised to match latest additions and some texts updated. Index: branches/Bungo/package/OptionalSettings/Resolution.txt =================================================================== --- branches/Bungo/package/OptionalSettings/Resolution.txt (revision 0) +++ branches/Bungo/package/OptionalSettings/Resolution.txt (revision 2840) @@ -0,0 +1,40 @@ +# --------------------------------------------- +# Chameleon Optional Settings List. +# --------------------------------------------- +# Add boot options or kernel flags to the bottom of this file. +# They will appear under the package installer's Settings menu +# in a sub menu named with the filename of this file. +# Use one file or many files - it's flexible to make it easy +# to group different options under separate sub menus. +# --------------------------------------------- +# To add boot option: Structure is: +# type@name:key=value +# example1: Bool@InstantMenu:Instant Menu=Yes +# example2: Text@1024x600x32:Graphics Mode=1024x600x32 +# example3: List@Npci:Kernel Flags=npci=0x2000 +# --------------------------------------------- +# type can be: Bool, Text or List +# --------------------------------------------- +# The package installer has a setting which controls what +# the user is allowed to choose. +# A) User can select every option from the list. +# B) User can select only one of the options from the list. +# Set Exclusive=False for A, or Exclusive=True for B. +# +Exclusive=True +# --------------------------------------------- +# Note: There must be a carriage return at end of last line +# --------------------------------------------- +Text@1024x600x32:Graphics Mode=1024x600x32 +Text@1024x768x32:Graphics Mode=1024x768x32 +Text@1280x768x32:Graphics Mode=1280x768x32 +Text@1280x800x32:Graphics Mode=1280x800x32 +Text@1280x960x32:Graphics Mode=1280x960x32 +Text@1280x1024x32:Graphics Mode=1280x1024x32 +Text@1366x768x32:Graphics Mode=1366x768x32 +Text@1440x900x32:Graphics Mode=1440x900x32 +Text@1600x900x32:Graphics Mode=1600x900x32 +Text@1600x1200x32:Graphics Mode=1600x1200x32 +Text@1680x1050x32:Graphics Mode=1680x1050x32 +Text@1920x1080x32:Graphics Mode=1920x1080x32 +Text@1920x1200x32:Graphics Mode=1920x1200x32 Index: branches/Bungo/package/OptionalSettings/Video.txt =================================================================== --- branches/Bungo/package/OptionalSettings/Video.txt (revision 0) +++ branches/Bungo/package/OptionalSettings/Video.txt (revision 2840) @@ -0,0 +1,37 @@ +# --------------------------------------------- +# Chameleon Optional Settings List. +# --------------------------------------------- +# Add boot options or kernel flags to the bottom of this file. +# They will appear under the package installer's Settings menu +# in a sub menu named with the filename of this file. +# Use one file or many files - it's flexible to make it easy +# to group different options under separate sub menus. +# --------------------------------------------- +# To add boot option: Structure is: +# type@name:key=value +# example1: Bool@InstantMenu:Instant Menu=Yes +# example2: Text@1024x600x32:Graphics Mode=1024x600x32 +# example3: List@Npci:Kernel Flags=npci=0x2000 +# --------------------------------------------- +# type can be: Bool, Text or List +# --------------------------------------------- +# The package installer has a setting which controls what +# the user is allowed to choose. +# A) User can select every option from the list. +# B) User can select only one of the options from the list. +# Set Exclusive=False for A, or Exclusive=True for B. +# +Exclusive=False +# --------------------------------------------- +# Note: There must be a carriage return at end of last line +# --------------------------------------------- +Bool@GraphicsEnabler:GraphicsEnabler=Yes +Bool@UseAtiROM:UseAtiROM=Yes +Bool@UseNvidiaROM:UseNvidiaROM=Yes +Bool@VBIOS:VBIOS=Yes +Bool@SkipIntelGfx:SkipIntelGfx=Yes +Bool@SkipNvidiaGfx:SkipNvidiaGfx=Yes +Bool@SkipAtiGfx:SkipAtiGfx=Yes +Bool@EnableBacklight:EnableBacklight=Yes +Bool@EnableDualLink:EnableDualLink=Yes +Bool@NvidiaGeneric:NvidiaGeneric=Yes Index: branches/Bungo/package/OptionalSettings/PowerManagement.txt =================================================================== --- branches/Bungo/package/OptionalSettings/PowerManagement.txt (revision 0) +++ branches/Bungo/package/OptionalSettings/PowerManagement.txt (revision 2840) @@ -0,0 +1,36 @@ +# --------------------------------------------- +# Chameleon Optional Settings List. +# --------------------------------------------- +# Add boot options or kernel flags to the bottom of this file. +# They will appear under the package installer's Settings menu +# in a sub menu named with the filename of this file. +# Use one file or many files - it's flexible to make it easy +# to group different options under separate sub menus. +# --------------------------------------------- +# To add boot option: Structure is: +# type@name:key=value +# example1: Bool@InstantMenu:Instant Menu=Yes +# example2: Text@1024x600x32:Graphics Mode=1024x600x32 +# example3: List@Npci:Kernel Flags=npci=0x2000 +# --------------------------------------------- +# type can be: Bool, Text or List +# --------------------------------------------- +# The package installer has a setting which controls what +# the user is allowed to choose. +# A) User can select every option from the list. +# B) User can select only one of the options from the list. +# Set Exclusive=False for A, or Exclusive=True for B. +# +Exclusive=False +# --------------------------------------------- +# Note: There must be a carriage return at end of last line +# --------------------------------------------- +Bool@CSTUsingSystemIO:CSTUsingSystemIO=Yes +Bool@DropSSDT:DropSSDT=Yes +Bool@EnableC2State:EnableC2State=Yes +Bool@EnableC3State:EnableC3State=Yes +Bool@EnableC4State:EnableC4State=Yes +#Bool@EnableC6State:EnableC6State=Yes +#Bool@EnableC7State:EnableC7State=Yes +Bool@GenerateCStates:GenerateCStates=Yes +Bool@GeneratePStates:GeneratePStates=Yes Index: branches/Bungo/package/OptionalSettings/General.txt =================================================================== --- branches/Bungo/package/OptionalSettings/General.txt (revision 0) +++ branches/Bungo/package/OptionalSettings/General.txt (revision 2840) @@ -0,0 +1,41 @@ +# --------------------------------------------- +# Chameleon Optional Settings List. +# --------------------------------------------- +# Add boot options or kernel flags to the bottom of this file. +# They will appear under the package installer's Settings menu +# in a sub menu named with the filename of this file. +# Use one file or many files - it's flexible to make it easy +# to group different options under separate sub menus. +# --------------------------------------------- +# To add boot option: Structure is: +# type@name:key=value +# example1: Bool@InstantMenu:Instant Menu=Yes +# example2: Text@1024x600x32:Graphics Mode=1024x600x32 +# example3: List@Npci:Kernel Flags=npci=0x2000 +# --------------------------------------------- +# type can be: Bool, Text or List +# --------------------------------------------- +# The package installer has a setting which controls what +# the user is allowed to choose. +# A) User can select every option from the list. +# B) User can select only one of the options from the list. +# Set Exclusive=False for A, or Exclusive=True for B. +# +Exclusive=False +# --------------------------------------------- +# Note: There must be a carriage return at end of last line +# --------------------------------------------- +Text@arch:arch=i386 +Bool@EHCIacquire:EHCIacquire=Yes +Bool@EthernetBuiltIn:EthernetBuiltIn=Yes +#Bool@EnableWifi:EnableWifi=Yes +Bool@ForceHPET:ForceHPET=Yes +Bool@ForceWake:ForceWake=Yes +Bool@ForceFullMemInfo:ForceFullMemInfo=Yes +Bool@RestartFix:RestartFix=No +Bool@UHCIreset:UHCIreset=Yes +Bool@XHCILegacyOff:XHCILegacyOff=Yes +Bool@UseMemDetect:UseMemDetect=No +Bool@UseKernelCache:UseKernelCache=Yes +Bool@Wake:Wake=Yes +Bool@PrivateData:PrivateData=No Index: branches/Bungo/package/OptionalSettings/KernelFlags.txt =================================================================== --- branches/Bungo/package/OptionalSettings/KernelFlags.txt (revision 0) +++ branches/Bungo/package/OptionalSettings/KernelFlags.txt (revision 2840) @@ -0,0 +1,37 @@ +# --------------------------------------------- +# Chameleon Optional Settings List. +# --------------------------------------------- +# Add boot options or kernel flags to the bottom of this file. +# They will appear under the package installer's Settings menu +# in a sub menu named with the filename of this file. +# Use one file or many files - it's flexible to make it easy +# to group different options under separate sub menus. +# --------------------------------------------- +# To add boot option: Structure is: +# type@name:key=value +# example1: Bool@InstantMenu:Instant Menu=Yes +# example2: Text@1024x600x32:Graphics Mode=1024x600x32 +# example3: List@Npci:Kernel Flags=npci=0x2000 +# --------------------------------------------- +# type can be: Bool, Text or List +# --------------------------------------------- +# The package installer has a setting which controls what +# the user is allowed to choose. +# A) User can select every option from the list. +# B) User can select only one of the options from the list. +# Set Exclusive=False for A, or Exclusive=True for B. +# +Exclusive=False +# --------------------------------------------- +# Note: There must be a carriage return at end of last line +# --------------------------------------------- +List@Verbose:Kernel Flags=-v +List@Singleusermode:Kernel Flags=-s +List@Ignorecaches:Kernel Flags=-f +List@Npci:Kernel Flags=npci=0x2000 +List@Npci3:Kernel Flags=npci=0x3000 +List@WaitingRootDevice:Kernel Flags=ahcidisk=1 debug=8 +List@Darkwake:Kernel Flags=darkwake=0 +List@NvdaDrv1:Kernel Flags=nvda_drv=1 +List@kext-dev-mode1:Kernel Flags=kext-dev-mode=1 +List@Dart0:Kernel Flags=dart=0 Index: branches/Bungo/package/OptionalSettings/Control.txt =================================================================== --- branches/Bungo/package/OptionalSettings/Control.txt (revision 0) +++ branches/Bungo/package/OptionalSettings/Control.txt (revision 2840) @@ -0,0 +1,34 @@ +# --------------------------------------------- +# Chameleon Optional Settings List. +# --------------------------------------------- +# Add boot options or kernel flags to the bottom of this file. +# They will appear under the package installer's Settings menu +# in a sub menu named with the filename of this file. +# Use one file or many files - it's flexible to make it easy +# to group different options under separate sub menus. +# --------------------------------------------- +# To add boot option: Structure is: +# type@name:key=value +# example1: Bool@InstantMenu:Instant Menu=Yes +# example2: Text@1024x600x32:Graphics Mode=1024x600x32 +# example3: List@Npci:Kernel Flags=npci=0x2000 +# --------------------------------------------- +# type can be: Bool, Text or List +# --------------------------------------------- +# The package installer has a setting which controls what +# the user is allowed to choose. +# A) User can select every option from the list. +# B) User can select only one of the options from the list. +# Set Exclusive=False for A, or Exclusive=True for B. +# +Exclusive=False +# --------------------------------------------- +# Note: There must be a carriage return at end of last line +# --------------------------------------------- +Bool@BootBanner:Boot Banner=No +Bool@GUI:GUI=No +Bool@LegacyLogo:Legacy Logo=Yes +Bool@InstantMenu:Instant Menu=Yes +Bool@QuietBoot:QuietBoot=Yes +Bool@ShowInfo:ShowInfo=Yes +Bool@Wait:Wait=Yes Index: branches/Bungo/package/smbios.plist =================================================================== --- branches/Bungo/package/smbios.plist (revision 0) +++ branches/Bungo/package/smbios.plist (revision 2840) @@ -0,0 +1,120 @@ + + + + + SMbiosvendor + Apple Inc. + SMbiosversion + MP41.88Z.0081.B08.1001221313 + SMbiosdate + 01/22/10 + SMmanufacturer + Apple Inc. + SMproductname + MacPro4,1 + SMsystemversion + 1.4 + SMserial + CK0215824PC + SMsystemuuid + a8dd17a0-cf10-4323-9841-29a9f08c1d49 + SMfamily + Mac Pro + SMboardmanufacturer + Apple Inc. + + SMboardproduct + Mac-F221BEC8 + SMboardversion + MacPro4,1 + SMboardserial + C02140302D5DMT31M + SMboardlocation + Part Component + + SMchassismanufacturer + Apple Inc. + SMchassisversion + Mac-F221BEC8 + SMchassisserial + CK0215824PC + + SMchassisassettag + Pro-Enclosure + SMexternalclock + 133 + SMmaximalclock + 3000 + + SMmemtype + 24 + SMmemspeed + 1600 + SMmemmanufacturer_1 + 0xAD00000000000000 + SMmemserial_1 + 0x00001020 + SMmempart_1 + 0x48594D503131325336344350362D59352020 + SMmemmanufacturer_2 + 0xAD00000000000000 + SMmemserial_2 + 0x00003021 + SMmempart_2 + 0x48594D503131325336344350362D59352020 + SMmemmanufacturer_3 + 0xAD00000000000000 + SMmemserial_3 + 0x00003021 + SMmempart_3 + 0x48594D503131325336344350362D59352020 + SMmemmanufacturer_4 + 0xAD00000000000000 + SMmemserial_4 + 0x00003021 + SMmempart_4 + 0x48594D503131325336344350362D59352020 + + + + Index: branches/Bungo/package/Scripts/Main/BootNO =================================================================== --- branches/Bungo/package/Scripts/Main/BootNO (revision 0) +++ branches/Bungo/package/Scripts/Main/BootNO (revision 2840) @@ -0,0 +1,8 @@ +#!/bin/bash + +configFile="/private/tmp/InstallConfig.plist" + +# adding option to declare that we are NOT installing the binary of the bootloader +# to the InstallConfig.plist +/usr/libexec/PlistBuddy -c "Add :bootloader bool false" ${configFile} + Property changes on: branches/Bungo/package/Scripts/Main/BootNO ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts/Main/BootYES =================================================================== --- branches/Bungo/package/Scripts/Main/BootYES (revision 0) +++ branches/Bungo/package/Scripts/Main/BootYES (revision 2840) @@ -0,0 +1,8 @@ +#!/bin/bash + +configFile="/private/tmp/InstallConfig.plist" + +# adding option to declare that we are NOT installing the binary of the bootloader +# to the InstallConfig.plist +/usr/libexec/PlistBuddy -c "Add :bootloader bool true" ${configFile} + Property changes on: branches/Bungo/package/Scripts/Main/BootYES ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts/Main/ESPpostinstall =================================================================== --- branches/Bungo/package/Scripts/Main/ESPpostinstall (revision 0) +++ branches/Bungo/package/Scripts/Main/ESPpostinstall (revision 2840) @@ -0,0 +1,418 @@ +#!/bin/bash + +# -------------------------------------------------------------------------------------------------------- +# Install.sh v1.3, script to install Chameleon +# Created by Miky1979 on December 8th, 2014 +# -------------------------------------------------------------------------------------------------------- +targetVolume="${3}" +InstallToESP="1" +InstallBootloader="0" +configFile="/private/tmp/InstallConfig.plist" + +if [[ $( /usr/libexec/plistbuddy -c "Print bootloader" ${configFile} ) == "true" ]];then + # installing stage 0, 1 and 2 only if user want this: + # ie only if have no selected noboot choice + InstallBootloader="1" +fi +# -------------------------------------------------------------------------------------------------------- +if [ ! -d "${targetVolume}" ]; then echo "target Volume not found, aborting!"; exit 1; fi +# -------------------------------------------------------------------------------------------------------- +i386Dir="${targetVolume}/usr/standalone/i386" +usrLocalBin="${targetVolume}/usr/local/bin" +# -------------------------------------------------------------------------------------------------------- +choicedVolume="${targetVolume}" +ESP_MOUNTED="0" +# -------------------------------------------------------------------------------------------------------- +v_mntptDev=$( /usr/libexec/plistbuddy -c "Print :ramdisk" ${configFile} | sed -e 's/[[:blank:]]*//g' ) +v_mntpt=$( LC_ALL=C diskutil info ${v_mntptDev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//') +backupRootDir="${targetVolume}/Chameleon.Backups" +backupDir=$( /usr/libexec/plistbuddy -c "Print :backupDir" ${configFile} ) +logName="Chameleon_Installer_Log.txt" +logFile="${v_mntpt}/${logName}" +# -------------------------------------------------------------------------------------------------------- +stage0Loader="boot0" +stage0LoaderDualBoot="boot0md" +stage1LoaderHFS="boot1h" +stage1LoaderFAT="boot1f32" +stage1LoaderExFAT="boot1x" +stage2Loader="boot" +versionToInstall=$(cat "${i386Dir}/Boot" | grep -a 'Darwin/x86 boot v' ) +# -------------------------------------------------------------------------------------------------------- +localdd="/bin/dd" +SS="${usrLocalBin}/sectorsize" +# -------------------------------------------------------------------------------------------------------- +# Scanning all Variables. We are switching to the target Volume or its ESP. +# This function is usefull to rescan the new target "at need" +SCAN() { + targetDevice=$( LC_ALL=C diskutil info "${choicedVolume}" | grep -i 'Device Node:' | awk '{print $NF}' ) + targetDeviceRaw=${targetDevice/disk/rdisk} + targetDisk=${targetDevice%s*} + targetDiskRaw=${targetDisk/disk/rdisk} + targetSlice=${targetDevice#*disk*s} + IOContent=$( LC_ALL=C diskutil info "${targetDisk}" | grep -i 'Content (IOContent)' | awk '{print $NF}' ) + FS=$( LC_ALL=C diskutil info ${targetDevice} | grep -i 'Type (Bundle)' | \ + awk '{print $NF}' | awk '{print tolower($0)}' ) + disksignature=$( dd 2>/dev/null if="${targetDisk}" count=1 | dd 2>/dev/null count=4 bs=1 skip=440 | \ + perl -ne '@a=split"";for(@a){printf"%02x",ord}' ) + + if [ $InstallBootloader = "1" ];then + blocksize=$( "${SS}" "${targetDeviceRaw}" | awk '{ print $2 }' ) + fi +} +# -------------------------------------------------------------------------------------------------------- +# lines +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" +# -------------------------------------------------------------------------------------------------------- +# Debug for Chameleon. +# Create a folder inside the Desktop called "DebugChameleon", and the log will be full +DEBUG() { + echo "$mainLine" + echo "DEBUG: display script variables (ESPpostinstall)" + echo "$mainLine" + echo "DEBUG: stage0Loader: Disk loader is ${stage0Loader} (or boot0hfs)" + echo "DEBUG: stage0LoaderDualBoot: Disk loader is ${stage0LoaderDualBoot} (or boot0hfs)" + echo "DEBUG: stage1LoaderHFS: Partition loader is ${stage1LoaderHFS}" + echo "DEBUG: stage1LoaderFat: Partition loader is ${stage1LoaderFAT}" + echo "DEBUG: stage1LoaderExFAT: Partition loader is ${stage1LoaderExFAT}" + echo "DEBUG: stage2Loader: Filesystem loader is ${stage2Loader}" + echo "DEBUG: targetVolume: Volume is ${targetVolume}" + echo "DEBUG: targetDevice: Volume device is ${targetDevice}" + echo "DEBUG: targetDeviceRaw: Volume raw device is ${targetDeviceRaw}" + echo "DEBUG: targetDisk: Disk device is ${targetDisk}" + echo "DEBUG: targetDiskRaw: Disk raw device is ${targetDiskRaw}" + echo "DEBUG: targetSlice: Volume slice is ${targetSlice}" + echo "DEBUG: versionToInstall: version to install is ${versionToInstall}" + echo "DEBUG: IOContent: partition scheme is ${IOContent}" + echo "DEBUG: FS: file System is ${FS}" + if [ $InstallBootloader = "1" ];then + echo "DEBUG: blocksize: block size detected = ${blocksize}-bytes" + fi +} +# -------------------------------------------------------------------------------------------------------- +# Checking for unsupported FAT16 filesystem +CHECK_FAT16() { + pers=$( LC_ALL=C diskutil info "${targetDeviceRaw}" | grep -i 'File System Personality' \ + | awk '{print $NF}' | awk '{print tolower($0)}' ) + + case $pers in + fat16) + echo "** ERROR: Can't Install to a device using FAT16!" + exit 1 + ;; + *) + echo "First Check Passed!" + ;; + esac +} +# -------------------------------------------------------------------------------------------------------- +# Mount or umount the ESP..or leave as is if already mounted.. +UMOUNT_ESP_DISK() { + # umount the ESP by its dev + umount -f $espDisk +} + +CHECK_ESP_MOUNTPOINT() { + # umount the ESP by its Mount Point + # and checking it if is busy by another ESP + if [ $( df | awk '{$1=$2=$3=$4=$5=$6=$7=$8=""; print $0}' | sed -e 's/^[ \t]*//' | \ + grep -x '/Volumes/ESP' ) ]; then + umount -f /Volumes/ESP + ESP_MOUNTED="0" + fi +} + +MOUNT_ESP() { + ESP_MOUNTED="1" # ..assuming that the ESP can be already mounted.. + if [ $( df | grep "${espDisk}" | awk '{print $1}' | grep -x "${espDisk}" ) ];then + # ESP is already mounted, so now we aquire the Mount Point + espmtp=$( LC_ALL=C diskutil info ${espDisk} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' | \ + sed -e 's/^[ \t]*//' ) + if [ -d "${espmtp}" ];then + echo "ESP Mount Point is:${espmtp}, using that as target Volume!" + choicedVolume="${espmtp}" + SCAN + else + echo "The mount point was not found, try to umount it to get the standard one.." + UMOUNT_ESP_DISK + CHECK_ESP_MOUNTPOINT + fi + else + # ESP is not mounted.. + ESP_MOUNTED="0" + fi + + if [ $ESP_MOUNTED = "0" ];then + mkdir -p /Volumes/ESP + case $( fstyp $espDisk ) in + hfs) + mount -t hfs $espDisk /Volumes/ESP + ;; + msdos) + mount -t msdos $espDisk /Volumes/ESP + ;; + *) + echo "ESP fileSystem unsupported, Installing to ${targetVolume}!" + choicedVolume="${targetVolume}" + ;; + esac + sleep 0.3 + + if [ $( df | awk '{$1=$2=$3=$4=$5=$6=$7=$8=""; print $0}' | sed -e 's/^[ \t]*//' | \ + grep -x '/Volumes/ESP' ) ]; then + ESP_MOUNTED="1" + choicedVolume="/Volumes/ESP" + SCAN + else + echo "ESP can't be mounted, Installing to ${targetVolume}!" + choicedVolume="${targetVolume}" + fi + fi +} +# -------------------------------------------------------------------------------------------------------- +PARTITION_ACTIVE_IF() { + echo -e "${mainLine}\nSET PARTITION ACTIVE:" + # if Windows was detected, don't activate the partition.. + # if the stage 0 loader is boo0hfs, don't activate the partition + if [ WINDOWS_EXIST = "0" ] || [ "${stage0Loader}" != "boot0hfs" ];then + partitionactive=$( fdisk -d ${targetDiskRaw} | grep -n "*" | awk -F: '{print $1}') + if [ "${partitionactive}" ] && [ "${partitionactive}" = "${targetSlice}" ]; then + echo "${targetDiskRaw#/dev/r}, slice "${targetSlice}" is already set active. No need to change it." + else + echo "Setting ${choicedVolume} partition active." + # BadAxe requires EFI partition to be flagged active. + # but it doesn't' hurt to do it for any non-windows partition. + +# leave left aligned the follow code: +fdisk -e ${targetDiskRaw} <<-MAKEACTIVE +print +flag ${targetSlice} +write +y +quit +MAKEACTIVE + fi + else + echo "Partition will not activate when Windows is detected or stage 0 is boot0hfs" + fi + echo "" + echo "$mainLine" +} +# -------------------------------------------------------------------------------------------------------- +# Writing stage 0 +CHECK_WINDOWS() { + WINDOWS_EXIST="0" + if [ "${disksignature}" = "00000000" ]; then + echo "Windows installation not found on ${targetDisk}." + else + echo "Windows installation found on ${targetDisk}." + WINDOWS_EXIST="1" + fi +} + +WRITE_STAGE0() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 0:" + CHECK_WINDOWS + case "$1" in + hfs) + if [ WINDOWS_EXIST = "1" ];then stage0Loader="boo0hfs"; else stage0Loader="boot0"; fi + ;; + msdos) + if [ WINDOWS_EXIST = "1" ];then stage0Loader="boo0md"; else stage0Loader="boot0";fi + ;; + exfat) + stage0Loader="boot0" + ;; + esac + + "${usrLocalBin}/fdisk440" -u -f "${i386Dir}/${stage0Loader}" -y ${targetDisk} + echo "${stage0Loader} writed to ${targetDisk}" + fi +} + +# -------------------------------------------------------------------------------------------------------- +# Writing stage 1 on different filesystems +WRITE_STAGE1_HFS() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 1 HFS:" + $localdd if="${i386Dir}/${stage1LoaderHFS}" of=${targetDeviceRaw} + echo "${stage1LoaderHFS} (hfs) writed to ${targetDeviceRaw}." + fi +} + +WRITE_STAGE1_EXFAT() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 1 ExFAT:" + cp -R "${usrLocalBin}/boot1-install" "${v_mntpt}"/ + cp -R "${i386Dir}/${stage1LoaderExFAT}" "${v_mntpt}"/ + "${v_mntpt}/boot1-install" -y -u -f "${v_mntpt}/${stage1LoaderExFAT}" ${targetDeviceRaw} + echo "${stage1LoaderExFAT} (ExFAT) writed to ${targetDeviceRaw}." + fi +} + +WRITE_STAGE1_FAT32() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 1 FAT32:" + $localdd if=${targetDeviceRaw} count=1 bs=512 of="${v_mntpt}/origbs" + cp "${i386Dir}/${stage1LoaderFAT}" "${v_mntpt}/newbs" + $localdd if="${v_mntpt}/origbs" of="${v_mntpt}/newbs" skip=3 seek=3 bs=1 count=87 conv=notrunc + $localdd if="${v_mntpt}/newbs" of="${targetDeviceRaw}" count=1 bs=512 + + echo "${stage1LoaderFAT} (Fat32) writed to ${targetDeviceRaw}." + fi +} +# -------------------------------------------------------------------------------------------------------- +# Writing stage 2 +WRITE_STAGE2() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 2:" + cp -R "${i386Dir}/${stage2Loader}" "${choicedVolume}/" + chflags hidden "${choicedVolume}/${stage2Loader}" + echo "Stage 2 (boot) writed to ${choicedVolume}." + fi +} +# -------------------------------------------------------------------------------------------------------- +# Waiting for targhet Volume was re-mounted before proceeding.. +# Note: the target Volume is umonted by boot1-install that also take the step to remount it (only waiting..) +WAIT_REMOUNT() { + if [ $InstallBootloader = "1" ];then + if [ ! -d "${choicedVolume}" ]; then + echo -e "${mainLine}\nWAITING TO RE-MOUNT ${choicedVolume}:" + until [ -d "${choicedVolume}" ]; do + sleep 0.3 + done + echo "${choicedVolume} is mounted!" + fi + fi +} +# -------------------------------------------------------------------------------------------------------- +# Extra folder... +EXTRA() { + echo -e "${mainLine}\nEXTRA FOLDER:" + if [ ! -d "${choicedVolume}/Extra" ]; then + echo "Extra not found on ${choicedVolume}, creating.." + mkdir -p "${v_mntpt}/Extra" + /usr/libexec/PlistBuddy -c "Add :'Kernel Flags' string -v" "${v_mntpt}/Extra/org.chameleon.Boot.plist" + else + echo "Extra folder already exist on ${choicedVolume}, copying to the Ram Disk.." + cp -R "${choicedVolume}/Extra" "${v_mntpt}"/ + ./clean_bootplist.pl "${v_mntpt}" >/dev/null + fi +} +# -------------------------------------------------------------------------------------------------------- +# Preparing Backing up of Chameleon files +BACKUP() { + echo -e "${mainLine}\nBACKUP CHAMELEON FILES:" + # Backup Stage 2 + if [ $InstallBootloader = "1" ];then + if [ -f "${choicedVolume}/boot" ]; then + echo "Backup stage2 file ${choicedVolume}/boot to ${backupRootDir}/${backupDir}/boot" + cp "${choicedVolume}/boot" "${backupRootDir}/${backupDir}/boot" + else + echo "No stage 2 (boot) was found, nothing to be saved." + fi + fi + + # Backup /Extra directory + if [[ -d "${choicedVolume}/Extra" ]];then + echo "Backing up ${choicedVolume}/Extra folder to ${backupRootDir}/${backupDir}/Extra" + cp -pR "${choicedVolume}/Extra" "${backupRootDir}/${backupDir}"/ + else + echo "No Extra folder was found, nothing to be saved." + fi +} +# -------------------------------------------------------------------------------------------------------- +if [ ! -d "${v_mntpt}" ]; then + echo "Ram Disk not found!" + exit +fi + +exec > >(tee -a "${logFile}") 2>&1 + +echo "$mainLine" +echo "Main ESP Post-Install Script" +echo "Chameleon installer log - $( date )" +if [ $InstallBootloader = "1" ];then echo "$versionToInstall"; else echo "no boot session"; fi +echo "" + +SCAN + +# (adjusted from Clover) +if [ ! "${targetDevice#*disk*s}" ]; then + echo + echo "*** ERROR: Volume does not use slices!" + echo + exit 1 +fi + +if [ "${InstallToESP}" == "1" ]; then + echo "$mainLine" + echo "SEARCHING ESP PARTITION:" + case "$IOContent" in + GUID_partition_scheme) + echo "GPT partition Scheme detected.." + espDisk="${targetDisk}s1" + if [ $( LC_ALL=C diskutil info ${espDisk} | grep -i 'Partition Type:' | \ + awk '{print $NF}' ) = "EFI" ]; then + echo "EFI partition found is ${espDisk}, try to mount it.." + MOUNT_ESP + else + echo "EFI was not fount, continue installing to ${targetVolume}" + choicedVolume="${targetVolume}" + fi + ;; + *) + echo "Can't install on the ESP, because does not exist.." + echo "..continue installing to ${targetVolume}" + ;; + esac +else + SCAN +fi + + +if [ -d "${HOME}/Desktop/DebugChameleon" ]; then + DEBUG +fi + +# adding the chosen Volume dev id to the InstallConfig.plist +/usr/libexec/PlistBuddy -c "Add :targetdev string ${targetDevice}" $configFile + +BACKUP +EXTRA + +echo "${mainLine}" +CHECK_FAT16 +case "$FS" in + hfs) + echo "${targetDevice} is HFS formatted" + WRITE_STAGE0 hfs + WRITE_STAGE1_HFS + ;; + msdos) + echo "${targetDevice} is FAT32 formatted" + WRITE_STAGE0 msdos + WRITE_STAGE1_FAT32 + ;; + exfat) + echo "${targetDevice} is ExFAT formatted" + WRITE_STAGE0 exfat + WRITE_STAGE1_EXFAT + WAIT_REMOUNT + ;; + *) + echo "FileSystem unsupported, aborting!" + exit 0 + ;; +esac + +WRITE_STAGE2 +PARTITION_ACTIVE_IF + +echo "$mainLine" +echo "END - ESP Post-Install Script" +# -------------------------------------------------------------------------------------------------------- + +exit 0 Property changes on: branches/Bungo/package/Scripts/Main/ESPpostinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/Scripts/Main/Standardpostinstall =================================================================== --- branches/Bungo/package/Scripts/Main/Standardpostinstall (revision 0) +++ branches/Bungo/package/Scripts/Main/Standardpostinstall (revision 2840) @@ -0,0 +1,417 @@ +#!/bin/bash + +# -------------------------------------------------------------------------------------------------------- +# Install.sh v1.3, script to install Chameleon +# Created by Miky1979 on December 8th, 2014 +# -------------------------------------------------------------------------------------------------------- +targetVolume="${3}" +InstallToESP="0" +InstallBootloader="0" +configFile="/private/tmp/InstallConfig.plist" + +if [[ $( /usr/libexec/plistbuddy -c "Print bootloader" ${configFile} ) == "true" ]];then + # installing stage 0, 1 and 2 only if user want this: + # ie only if have no selected noboot choice + InstallBootloader="1" +fi +# -------------------------------------------------------------------------------------------------------- +if [ ! -d "${targetVolume}" ]; then echo "target Volume not found, aborting!"; exit 1; fi +# -------------------------------------------------------------------------------------------------------- +i386Dir="${targetVolume}/usr/standalone/i386" +usrLocalBin="${targetVolume}/usr/local/bin" +# -------------------------------------------------------------------------------------------------------- +choicedVolume="${targetVolume}" +ESP_MOUNTED="0" +# -------------------------------------------------------------------------------------------------------- +v_mntptDev=$( /usr/libexec/plistbuddy -c "Print :ramdisk" ${configFile} | sed -e 's/[[:blank:]]*//g' ) +v_mntpt=$( LC_ALL=C diskutil info ${v_mntptDev} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' \ + | sed -e 's/^[ \t]*//') +backupRootDir="${targetVolume}/Chameleon.Backups" +backupDir=$( /usr/libexec/plistbuddy -c "Print :backupDir" ${configFile} ) +logName="Chameleon_Installer_Log.txt" +logFile="${v_mntpt}/${logName}" +# -------------------------------------------------------------------------------------------------------- +stage0Loader="boot0" +stage0LoaderDualBoot="boot0md" +stage1LoaderHFS="boot1h" +stage1LoaderFAT="boot1f32" +stage1LoaderExFAT="boot1x" +stage2Loader="boot" +versionToInstall=$(cat "${i386Dir}/Boot" | grep -a 'Darwin/x86 boot v' ) +# -------------------------------------------------------------------------------------------------------- +localdd="/bin/dd" +SS="${usrLocalBin}/sectorsize" +# -------------------------------------------------------------------------------------------------------- +# Scanning all Variables. We are switching to the target Volume or its ESP. +# This function is usefull to rescan the new target "at need" +SCAN() { + targetDevice=$( LC_ALL=C diskutil info "${choicedVolume}" | grep -i 'Device Node:' | awk '{print $NF}' ) + targetDeviceRaw=${targetDevice/disk/rdisk} + targetDisk=${targetDevice%s*} + targetDiskRaw=${targetDisk/disk/rdisk} + targetSlice=${targetDevice#*disk*s} + IOContent=$( LC_ALL=C diskutil info "${targetDisk}" | grep -i 'Content (IOContent)' | awk '{print $NF}' ) + FS=$( LC_ALL=C diskutil info ${targetDevice} | grep -i 'Type (Bundle)' | \ + awk '{print $NF}' | awk '{print tolower($0)}' ) + disksignature=$( dd 2>/dev/null if="${targetDisk}" count=1 | dd 2>/dev/null count=4 bs=1 skip=440 | \ + perl -ne '@a=split"";for(@a){printf"%02x",ord}' ) + + if [ $InstallBootloader = "1" ];then + blocksize=$( "${SS}" "${targetDeviceRaw}" | awk '{ print $2 }' ) + fi +} +# -------------------------------------------------------------------------------------------------------- +# lines +mainLine="==============================================================================" +subLine="------------------------------------------------------------------------------" +# -------------------------------------------------------------------------------------------------------- +# Debug for Chameleon. +# Create a folder inside the Desktop called "DebugChameleon", and the log will be full +DEBUG() { + echo "$mainLine" + echo "DEBUG: display script variables (Standardpostinstall)" + echo "$mainLine" + echo "DEBUG: stage0Loader: Disk loader is ${stage0Loader} (or boot0hfs)" + echo "DEBUG: stage0LoaderDualBoot: Disk loader is ${stage0LoaderDualBoot} (or boot0hfs)" + echo "DEBUG: stage1LoaderHFS: Partition loader is ${stage1LoaderHFS}" + echo "DEBUG: stage1LoaderFat: Partition loader is ${stage1LoaderFAT}" + echo "DEBUG: stage1LoaderExFAT: Partition loader is ${stage1LoaderExFAT}" + echo "DEBUG: stage2Loader: Filesystem loader is ${stage2Loader}" + echo "DEBUG: targetVolume: Volume is ${targetVolume}" + echo "DEBUG: targetDevice: Volume device is ${targetDevice}" + echo "DEBUG: targetDeviceRaw: Volume raw device is ${targetDeviceRaw}" + echo "DEBUG: targetDisk: Disk device is ${targetDisk}" + echo "DEBUG: targetDiskRaw: Disk raw device is ${targetDiskRaw}" + echo "DEBUG: targetSlice: Volume slice is ${targetSlice}" + echo "DEBUG: versionToInstall: version to install is ${versionToInstall}" + echo "DEBUG: IOContent: partition scheme is ${IOContent}" + echo "DEBUG: FS: file System is ${FS}" + if [ $InstallBootloader = "1" ];then + echo "DEBUG: blocksize: block size detected = ${blocksize}-bytes" + fi +} +# -------------------------------------------------------------------------------------------------------- +# Checking for unsupported FAT16 filesystem +CHECK_FAT16() { + pers=$( LC_ALL=C diskutil info "${targetDeviceRaw}" | grep -i 'File System Personality' \ + | awk '{print $NF}' | awk '{print tolower($0)}' ) + + case $pers in + fat16) + echo "** ERROR: Can't Install to a device using FAT16!" + exit 1 + ;; + *) + echo "First Check Passed!" + ;; + esac +} +# -------------------------------------------------------------------------------------------------------- +# Mount or umount the ESP..or leave as is if already mounted.. +UMOUNT_ESP_DISK() { + # umount the ESP by its dev + umount -f $espDisk +} + +CHECK_ESP_MOUNTPOINT() { + # umount the ESP by its Mount Point + # and checking it if is busy by another ESP + if [ $( df | awk '{$1=$2=$3=$4=$5=$6=$7=$8=""; print $0}' | sed -e 's/^[ \t]*//' | \ + grep -x '/Volumes/ESP' ) ]; then + umount -f /Volumes/ESP + ESP_MOUNTED="0" + fi +} + +MOUNT_ESP() { + ESP_MOUNTED="1" # ..assuming that the ESP can be already mounted.. + if [ $( df | grep "${espDisk}" | awk '{print $1}' | grep -x "${espDisk}" ) ];then + # ESP is already mounted, so now we aquire the Mount Point + espmtp=$( LC_ALL=C diskutil info ${espDisk} | grep -i 'mount point' | awk '{$1=$2=""; print $0}' | \ + sed -e 's/^[ \t]*//' ) + if [ -d "${espmtp}" ];then + echo "ESP Mount Point is:${espmtp}, using that as target Volume!" + choicedVolume="${espmtp}" + SCAN + else + echo "The mount point was not found, try to umount it to get the standard one.." + UMOUNT_ESP_DISK + CHECK_ESP_MOUNTPOINT + fi + else + # ESP is not mounted.. + ESP_MOUNTED="0" + fi + + if [ $ESP_MOUNTED = "0" ];then + mkdir -p /Volumes/ESP + case $( fstyp $espDisk ) in + hfs) + mount -t hfs $espDisk /Volumes/ESP + ;; + msdos) + mount -t msdos $espDisk /Volumes/ESP + ;; + *) + echo "ESP fileSystem unsupported, Installing to ${targetVolume}!" + choicedVolume="${targetVolume}" + ;; + esac + sleep 0.3 + + if [ $( df | awk '{$1=$2=$3=$4=$5=$6=$7=$8=""; print $0}' | sed -e 's/^[ \t]*//' | \ + grep -x '/Volumes/ESP' ) ]; then + ESP_MOUNTED="1" + choicedVolume="/Volumes/ESP" + SCAN + else + echo "ESP can't be mounted, Installing to ${targetVolume}!" + choicedVolume="${targetVolume}" + fi + fi +} +# -------------------------------------------------------------------------------------------------------- +PARTITION_ACTIVE_IF() { + echo -e "${mainLine}\nSET PARTITION ACTIVE:" + # if Windows was detected, don't activate the partition.. + # if the stage 0 loader is boo0hfs, don't activate the partition + if [ WINDOWS_EXIST = "0" ] || [ "${stage0Loader}" != "boot0hfs" ];then + partitionactive=$( fdisk -d ${targetDiskRaw} | grep -n "*" | awk -F: '{print $1}') + if [ "${partitionactive}" ] && [ "${partitionactive}" = "${targetSlice}" ]; then + echo "${targetDiskRaw#/dev/r}, slice "${targetSlice}" is already set active. No need to change it." + else + echo "Setting ${choicedVolume} partition active." + # BadAxe requires EFI partition to be flagged active. + # but it doesn't' hurt to do it for any non-windows partition. + +# leave left aligned the follow code: +fdisk -e ${targetDiskRaw} <<-MAKEACTIVE +print +flag ${targetSlice} +write +y +quit +MAKEACTIVE + fi + else + echo "Partition will not activate when Windows is detected or stage 0 is boot0hfs" + fi + echo "" + echo "$mainLine" +} +# -------------------------------------------------------------------------------------------------------- +# Writing stage 0 +CHECK_WINDOWS() { + WINDOWS_EXIST="0" + if [ "${disksignature}" = "00000000" ]; then + echo "Windows installation not found on ${targetDisk}." + else + echo "Windows installation found on ${targetDisk}." + WINDOWS_EXIST="1" + fi +} + +WRITE_STAGE0() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 0:" + CHECK_WINDOWS + case "$1" in + hfs) + if [ WINDOWS_EXIST = "1" ];then stage0Loader="boo0hfs"; else stage0Loader="boot0"; fi + ;; + msdos) + if [ WINDOWS_EXIST = "1" ];then stage0Loader="boo0md"; else stage0Loader="boot0";fi + ;; + exfat) + stage0Loader="boot0" + ;; + esac + + "${usrLocalBin}/fdisk440" -u -f "${i386Dir}/${stage0Loader}" -y ${targetDisk} + echo "${stage0Loader} writed to ${targetDisk}" + fi +} +# -------------------------------------------------------------------------------------------------------- +# Writing stage 1 on different filesystems +WRITE_STAGE1_HFS() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 1 HFS:" + $localdd if="${i386Dir}/${stage1LoaderHFS}" of=${targetDeviceRaw} + echo "${stage1LoaderHFS} (hfs) writed to ${targetDeviceRaw}." + fi +} + +WRITE_STAGE1_EXFAT() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 1 ExFAT:" + cp -R "${usrLocalBin}/boot1-install" "${v_mntpt}"/ + cp -R "${i386Dir}/${stage1LoaderExFAT}" "${v_mntpt}"/ + "${v_mntpt}/boot1-install" -y -u -f "${v_mntpt}/${stage1LoaderExFAT}" ${targetDeviceRaw} + echo "${stage1LoaderExFAT} (ExFAT) writed to ${targetDeviceRaw}." + fi +} + +WRITE_STAGE1_FAT32() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 1 FAT32:" + $localdd if=${targetDeviceRaw} count=1 bs=512 of="${v_mntpt}/origbs" + cp "${i386Dir}/${stage1LoaderFAT}" "${v_mntpt}/newbs" + $localdd if="${v_mntpt}/origbs" of="${v_mntpt}/newbs" skip=3 seek=3 bs=1 count=87 conv=notrunc + $localdd if="${v_mntpt}/newbs" of="${targetDeviceRaw}" count=1 bs=512 + + echo "${stage1LoaderFAT} (Fat32) writed to ${targetDeviceRaw}." + fi +} +# -------------------------------------------------------------------------------------------------------- +# Writing stage 2 +WRITE_STAGE2() { + if [ $InstallBootloader = "1" ];then + echo -e "${mainLine}\nWRITING STAGE 2:" + cp -R "${i386Dir}/${stage2Loader}" "${choicedVolume}/" + chflags hidden "${choicedVolume}/${stage2Loader}" + echo "Stage 2 (boot) writed to ${choicedVolume}." + fi +} +# -------------------------------------------------------------------------------------------------------- +# Waiting for targhet Volume was re-mounted before proceeding.. +# Note: the target Volume is umonted by boot1-install that also take the step to remount it (only waiting..) +WAIT_REMOUNT() { + if [ $InstallBootloader = "1" ];then + if [ ! -d "${choicedVolume}" ]; then + echo -e "${mainLine}\nWAITING TO RE-MOUNT ${choicedVolume}:" + until [ -d "${choicedVolume}" ]; do + sleep 0.3 + done + echo "${choicedVolume} is mounted!" + fi + fi +} +# -------------------------------------------------------------------------------------------------------- +# Extra folder... +EXTRA() { + echo -e "${mainLine}\nEXTRA FOLDER:" + if [ ! -d "${choicedVolume}/Extra" ]; then + echo "Extra not found on ${choicedVolume}, creating.." + mkdir -p "${v_mntpt}/Extra" + /usr/libexec/PlistBuddy -c "Add :'Kernel Flags' string -v" "${v_mntpt}/Extra/org.chameleon.Boot.plist" + else + echo "Extra folder already exist on ${choicedVolume}, copying to the Ram Disk.." + cp -R "${choicedVolume}/Extra" "${v_mntpt}"/ + ./clean_bootplist.pl "${v_mntpt}" >/dev/null + fi +} +# -------------------------------------------------------------------------------------------------------- +# Preparing Backing up of Chameleon files +BACKUP() { + echo -e "${mainLine}\nBACKUP CHAMELEON FILES:" + # Backup Stage 2 + if [ $InstallBootloader = "1" ];then + if [ -f "${choicedVolume}/boot" ]; then + echo "Backup stage2 file ${choicedVolume}/boot to ${backupRootDir}/${backupDir}/boot" + cp "${choicedVolume}/boot" "${backupRootDir}/${backupDir}/boot" + else + echo "No stage 2 (boot) was found, nothing to be saved." + fi + fi + + # Backup /Extra directory + if [[ -d "${choicedVolume}/Extra" ]];then + echo "Backing up ${choicedVolume}/Extra folder to ${backupRootDir}/${backupDir}/Extra" + cp -pR "${choicedVolume}/Extra" "${backupRootDir}/${backupDir}"/ + else + echo "No Extra folder was found, nothing to be saved." + fi +} +# -------------------------------------------------------------------------------------------------------- +if [ ! -d "${v_mntpt}" ]; then + echo "Ram Disk not found!" + exit +fi + +exec > >(tee -a "${logFile}") 2>&1 + +echo "$mainLine" +echo "Main Standard Post-Install Script" +echo "Chameleon installer log - $( date )" +if [ $InstallBootloader = "1" ];then echo "$versionToInstall"; else echo "no boot session"; fi +echo "" + +SCAN + +# (adjusted from Clover) +if [ ! "${targetDevice#*disk*s}" ]; then + echo + echo "*** ERROR: Volume does not use slices!" + echo + exit 1 +fi + +if [ "${InstallToESP}" == "1" ]; then + echo "$mainLine" + echo "SEARCHING ESP PARTITION:" + case "$IOContent" in + GUID_partition_scheme) + echo "GPT partition Scheme detected.." + espDisk="${targetDisk}s1" + if [ $( LC_ALL=C diskutil info ${espDisk} | grep -i 'Partition Type:' | \ + awk '{print $NF}' ) = "EFI" ]; then + echo "EFI partition found is ${espDisk}, try to mount it.." + MOUNT_ESP + else + echo "EFI was not fount, continue installing to ${targetVolume}" + choicedVolume="${targetVolume}" + fi + ;; + *) + echo "Can't install on the ESP, because does not exist.." + echo "..continue installing to ${targetVolume}" + ;; + esac +else + SCAN +fi + + +if [ -d "${HOME}/Desktop/DebugChameleon" ]; then + DEBUG +fi + +# adding the chosen Volume dev id to the InstallConfig.plist +/usr/libexec/PlistBuddy -c "Add :targetdev string ${targetDevice}" $configFile + +BACKUP +EXTRA + +echo "${mainLine}" +CHECK_FAT16 +case "$FS" in + hfs) + echo "${targetDevice} is HFS formatted" + WRITE_STAGE0 hfs + WRITE_STAGE1_HFS + ;; + msdos) + echo "${targetDevice} is FAT32 formatted" + WRITE_STAGE0 msdos + WRITE_STAGE1_FAT32 + ;; + exfat) + echo "${targetDevice} is ExFAT formatted" + WRITE_STAGE0 exfat + WRITE_STAGE1_EXFAT + WAIT_REMOUNT + ;; + *) + echo "FileSystem unsupported, aborting!" + exit 0 + ;; +esac + +WRITE_STAGE2 +PARTITION_ACTIVE_IF + +echo "$mainLine" +echo "END - Standard Post-Install Script" +# -------------------------------------------------------------------------------------------------------- + +exit 0 Property changes on: branches/Bungo/package/Scripts/Main/Standardpostinstall ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/bin/po4a/po4a-normalize =================================================================== --- branches/Bungo/package/bin/po4a/po4a-normalize (revision 0) +++ branches/Bungo/package/bin/po4a/po4a-normalize (revision 2840) @@ -0,0 +1,181 @@ +#! /usr/bin/env perl +eval 'exec perl -S $0 ${1+"$@"}' + if $running_under_some_shell; + +# po4a-normalize -- normalize documentation files +# +# Copyright 2002-2012 by SPI, inc. +# +# This program is free software; you can redistribute it and/or modify it +# under the terms of GPL (see COPYING). + +=encoding UTF-8 + +=head1 NAME + +po4a-normalize - normalize a documentation file by parsing it in po4a, and writing it back + +=head1 SYNOPSIS + +B B<-f> I I + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +The B script is a debugging tool used to make sure that +po4a don't change the document when it's not supposed to. Only use it if +you're developing a new module, or if you doubt the sanity of the tools. + +The generated document will be written to F while the +generated PO file will be written to F. No way to change +that ;) + +=head1 OPTIONS + +=over 4 + +=item B<-o>, B<--option> + +Extra option(s) to pass to the format plugin. Specify each option in the +'IB<=>I' format. See the documentation of each plugin for more +information about the valid options and their meanings. + +=item B<-b>, B<--blank> + +Create an blank translated document. +The generated translated document will be generated assuming all messages +are translated by a space or new line. + +This is useful to check what parts of the document cannot be translated. + +=item B<-h>, B<--help> + +Show a short help message. + +=item B<--help-format> + +List the documentation formats understood by po4a. + +=item B<-f>, B<--format> + +Format of the documentation you want to handle. Use the B<--help-format> +option to see the list of available formats. + +=item B<-M>, B<--master-charset> + +Charset of the file containing the document to translate. + +=item B<-V>, B<--version> + +Display the version of the script and exit. + +=back + +=head1 SEE ALSO + +L, +L, +L, +L + +=head1 AUTHORS + + Denis Barbier + Nicolas François + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002-2012 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +use 5.006; +use strict; +use warnings; + +use Locale::Po4a::Chooser; +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +use Getopt::Long qw(GetOptions); + +use Pod::Usage qw(pod2usage); + +Locale::Po4a::Common::textdomain('po4a'); + +sub show_version { + Locale::Po4a::Common::show_version("po4a-normalize"); + exit 0; +} + +my ($blank,$help_fmt,$help,$type,$debug,$verbose,@options); +my ($mastchar); +Getopt::Long::Configure('no_auto_abbrev','no_ignore_case'); +GetOptions( + 'help|h' => \$help, + 'help-format' => \$help_fmt, + 'format|f=s' => \$type, + + 'blank|b' => \$blank, + + 'master-charset|M=s' => \$mastchar, + + 'option|o=s' => \@options, + + 'verbose|v' => \$verbose, + 'debug|d' => \$debug, + 'version|V' => \&show_version +) or pod2usage(); + +$help && pod2usage (-verbose => 1, -exitval => 0); +$help_fmt && Locale::Po4a::Chooser::list(0); +pod2usage () unless scalar @ARGV == 1; + +my %options = ( + "verbose" => $verbose, + "debug" => $debug); +foreach (@options) { + if (m/^([^=]*)=(.*)$/) { + $options{$1}="$2"; + } else { + $options{$_}=1; + } +} + +my $parser=Locale::Po4a::Chooser::new($type,%options); + +my $filename = shift || pod2usage(1); +$filename eq '-' || -e $filename || die wrap_msg(gettext("File %s does not exist."), $filename); + +$parser->read($filename); +$parser->{TT}{utf_mode} = 1; +$parser->{TT}{file_in_charset} = $mastchar; +$parser->parse(); +if ($blank) { + foreach my $msgid (keys %{$parser->{TT}{po_out}{po}}) { + if ($msgid =~ m/\n$/s) { + $parser->{TT}{po_out}{po}{$msgid}{'msgstr'} = "\n"; + } else { + $parser->{TT}{po_out}{po}{$msgid}{'msgstr'} = " "; + } + } + my $empty_po = $parser->{TT}{po_out}; + $parser = Locale::Po4a::Chooser::new($type,%options); + $parser->{TT}{po_in} = $empty_po; + $parser->read($filename); + $parser->{TT}{utf_mode} = 1; + $parser->{TT}{file_in_charset} = $mastchar; + $parser->parse(); +} +$parser->write('po4a-normalize.output'); +$parser->writepo('po4a-normalize.po'); + +__END__ + Property changes on: branches/Bungo/package/bin/po4a/po4a-normalize ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/bin/po4a/po4a-updatepo =================================================================== --- branches/Bungo/package/bin/po4a/po4a-updatepo (revision 0) +++ branches/Bungo/package/bin/po4a/po4a-updatepo (revision 2840) @@ -0,0 +1,289 @@ +#! /usr/bin/env perl +eval 'exec perl -S $0 ${1+"$@"}' + if $running_under_some_shell; + +# pod-updatepo -- update the PO translation of POD data. +# +# Copyright 2002-2012 by SPI, inc. +# +# This program is free software; you can redistribute it and/or modify it +# under the terms of GPL (see COPYING). + +=encoding UTF-8 + +=head1 NAME + +po4a-updatepo - update the translation (in PO format) of documentation + +=head1 SYNOPSIS + +B B<-f> I (B<-m> I)+ (B<-p> I)+ + +(I are the outputs, all others are inputs) + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +The B script is in charge of updating PO files to make +them reflect the changes made to the original documentation file. For that, +it converts the documentation file to a POT file, and call L +on this new POT and on the provided PO files. + +It is possible to give more than one PO file (if you want to update several +languages at once), and several documentation files (if you want to store +the translations of several documents in the same PO file). + +If the master document has non-ASCII characters, it will convert the PO files +to UTF-8 (if they weren't already), in order to allow non-standard characters +in a culture independent way. + +=head1 OPTIONS + +=over 4 + +=item B<-f>, B<--format> + +Format of the documentation you want to handle. Use the B<--help-format> +option to see the list of available formats. + +=item B<-m>, B<--master> + +File(s) containing the master document to translate. + +=item B<-M>, B<--master-charset> + +Charset of the files containing the document to translate. Note that all +files must have the same charset. + +=item B<-p>, B<--po> + +PO file(s) to update. If these files do not exist, they are created by +B. + +=item B<-o>, B<--option> + +Extra option(s) to pass to the format plugin and other po4a internal module. +Specify each option in the 'IB<=>I' format. See the documentation of +each plugin for more information about the valid options and their meanings. + +=item B<--no-previous> + +This option removes B<--previous> from the options passed to B. +This permits to support versions of B earlier than 0.16. + +=item B<--previous> + +This option adds B<--previous> to the options passed to B. +It requires B 0.16 or later, and is activated by default. + +=item B<--msgmerge-opt> I + +Extra options for B. + +=item B<-h>, B<--help> + +Show a short help message. + +=item B<--help-format> + +List the documentation formats understood by po4a. + +=item B<-V>, B<--version> + +Display the version of the script and exit. + +=item B<-v>, B<--verbose> + +Increase the verbosity of the program. + +=item B<-d>, B<--debug> + +Output some debugging information. + +=item B I[,B|B] + +Specify the reference format. Argument I can be one of B to not +produce any reference, B to not specify the line number (more +accurately all line numbers are replaced by 1), B to replace line +number by an increasing counter, and B to include complete +references. + +Argument can be followed by a comma and either B or B keyword. +References are written by default on a single line. The B option wraps +references on several lines, to mimic B tools (B and +B). This option will become the default in a future release, because +it is more sensible. The B option is available so that users who want +to keep the old behavior can do so. + +=item B<--msgid-bugs-address> I + +Set the report address for msgid bugs. By default, the created POT files +have no Report-Msgid-Bugs-To fields. + +=item B<--copyright-holder> I + +Set the copyright holder in the POT header. The default value is +"Free Software Foundation, Inc." + +=item B<--package-name> I + +Set the package name for the POT header. The default is "PACKAGE". + +=item B<--package-version> I + +Set the package version for the POT header. The default is "VERSION". + +=back + +=head1 SEE ALSO + +L, +L, +L, +L + +=head1 AUTHORS + + Denis Barbier + Nicolas François + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002-2012 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +use 5.006; +use strict; +use warnings; + +use Getopt::Long qw(GetOptions); +use Locale::Po4a::Po; + +use Locale::Po4a::Chooser; +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +use Pod::Usage qw(pod2usage); + +use File::Temp; + +Locale::Po4a::Common::textdomain('po4a'); + +sub show_version { + Locale::Po4a::Common::show_version("po4a-updatepo"); + exit 0; +} + + +# init commandline parser +Getopt::Long::config('bundling', 'no_getopt_compat', 'no_auto_abbrev'); + +# Parse our options +my (@masterfiles,@pofiles); +my ($help,$help_fmt,$verbose,$debug,$format,@options); +my ($copyright_holder, $msgid_bugs_address, $package_name, $package_version); +my $mastchar; +my $previous; +my $noprevious; +my $msgmerge_opt = ""; +GetOptions('help|h' => \$help, + 'help-format' => \$help_fmt, + + 'master|m=s' => \@masterfiles, + 'po|p=s' => \@pofiles, + 'format|f=s' => \$format, + + 'master-charset|M=s' => \$mastchar, + + 'option|o=s' => \@options, + + 'no-previous' => \$noprevious, + 'previous' => \$previous, + 'msgmerge-opt=s' => \$msgmerge_opt, + 'copyright-holder=s' => \$copyright_holder, + 'msgid-bugs-address=s'=> \$msgid_bugs_address, + 'package-name=s' => \$package_name, + 'package-version=s' => \$package_version, + + 'verbose|v' => \$verbose, + 'debug|d' => \$debug, + 'version|V' => \&show_version) + or pod2usage(); + +$help && pod2usage (-verbose => 1, -exitval => 0); +$help_fmt && Locale::Po4a::Chooser::list(0); +pod2usage () if scalar @masterfiles < 1 || scalar @pofiles < 1; + +$msgmerge_opt .= " --previous" unless $noprevious; + +my %options = ( + "verbose" => $verbose, + "debug" => $debug, + "copyright-holder" => $copyright_holder, + "msgid-bugs-address" => $msgid_bugs_address, + "package-name" => $package_name, + "package-version" => $package_version); + +foreach (@options) { + if (m/^([^=]*)=(.*)$/) { + $options{$1}="$2"; + } else { + $options{$_}=1; + } +} + +# parser +my ($doc)=Locale::Po4a::Chooser::new($format,%options); + +map { -e $_ || die wrap_msg(gettext("File %s does not exist."), $_) } @masterfiles; +map { die wrap_msg(gettext("po4a-updatepo can't take the input PO from stdin.")) + if $_ eq '-' && !-e '-'} @pofiles; + +my ($pot_filename); +(undef,$pot_filename)=File::Temp->tempfile("po4a-updatepoXXXX", + DIR => "/tmp", + SUFFIX => ".pot", + OPEN => 0, + UNLINK => 0) + or die wrap_msg(gettext("Can't create a temporary POT file: %s"), $!); + + +print STDERR wrap_msg(gettext("Parse input files... ")) if $verbose; + +$doc->{TT}{utf_mode} = 1; + +$doc->process('file_in_name' => \@masterfiles, + 'file_in_charset' => $mastchar, + 'po_out_name' => $pot_filename, + 'debug' => $debug, + 'verbose' => $verbose); + +print STDERR wrap_msg(gettext("done.")) if $verbose; + + +while (my $po_filename=shift @pofiles) { + if (-e $po_filename) { + print STDERR wrap_msg(gettext("Updating %s:"), $po_filename) + if $verbose; + my $cmd = "msgmerge $msgmerge_opt -U $po_filename $pot_filename"; + system ($cmd) == 0 + or die wrap_msg(gettext("Error while running msgmerge: %s"), $!); + system "msgfmt --statistics -v -o /dev/null $po_filename" + if $verbose; + } else { + print STDERR wrap_msg(gettext("Creating %s:"), $po_filename) + if $verbose; + system ("cp",$pot_filename,$po_filename) == 0 + or die wrap_msg(gettext("Error while copying the PO file: %s"), $!); + } +} + +unlink($pot_filename); Property changes on: branches/Bungo/package/bin/po4a/po4a-updatepo ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/bin/po4a/po4a =================================================================== --- branches/Bungo/package/bin/po4a/po4a (revision 0) +++ branches/Bungo/package/bin/po4a/po4a (revision 2840) @@ -0,0 +1,1584 @@ +#! /usr/bin/env perl +eval 'exec perl -S $0 ${1+"$@"}' + if $running_under_some_shell; + +# po4a -- Update both the po files and translated documents in one shoot +# +# Copyright 2002-2012 by SPI, inc. +# +# This program is free software; you can redistribute it and/or modify it +# under the terms of GPL (see COPYING). + +=encoding UTF-8 + +=head1 NAME + +po4a - update both the PO files and translated documents in one shot + +=head1 SYNOPSIS + +B [I] I + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +The B program is useful if you want to avoid calling +L, L, and L in +complex Makefiles when you have multiple files to translate, different +format, or need to specify different options for different documents. + +=head1 Table of content + +This document is organized as follow: + +=head2 DESCRIPTION + +=head2 INTRODUCTION + +=head2 CONFIGURATION FILE SYNTAX + +=head3 Specifying the template languages + +=head3 Specifying the paths to translator inputs + +=head3 Autodetection of the paths and languages + +=head3 Specifying the documents to translate + +=head3 Specifying options for the modules + +=head3 Specifying aliases + +=head3 Split mode + +=head2 OPTIONS + +=head2 EXAMPLE + +=head2 SHORTCOMINGS + +=head2 SEE ALSO + +=head2 AUTHORS + +=head2 COPYRIGHT AND LICENSE + +=head3 + +=head1 INTRODUCTION + +The B program is in charge of updating both the PO files (to sync +them to the original documents) and the translated documents (to sync +them to the PO files). The main point is to make the use of po4a easier +without having to remember of the command line options. + +It also allows you to mix documents having different formats into the same +POT file so that you can have only one such file per project. + +This behaviour can be mimicked by the other tools of the po4a suite (for +example with Makefiles), but it is rather difficult to do, and exhausting to +redo the same complicated Makefiles for each project using po4a. + +The dataflow can be summarized as follow. Any changes to the master document +will be reflected in the PO files, and all changes to the PO files (either +manual or caused by previous step) will be reflected in translation +documents. + + master document --> PO files --> translations + +The dataflow cannot be inversed in this tool, and changes in translations +are overwritten by the content of the PO files. As a matter of fact, this +tool cannot be used to convert existing translations to the po4a system. For +that task, please refer to L. + +=head1 CONFIGURATION FILE SYNTAX + +The (mandatory) argument is the path to the configuration file to use. +Its syntax aims at being simple and close to the configuration files used +by the intl-tools projects. + +Comments in this files are noted by the char '#'. It comments everything +until the end of the line. Lines can be continued by escaping the end of line. +All non blank lines must begin with a [] command, followed by its arguments. +(sound difficult said that way, but it is rather easy, I hope ;) + +=head2 Specifying the template languages + +B It is recommended to use B<[po_directory]> rather than B<[po4a_langs]> +and B<[po4a_paths]>. +See section B below. + +This is an optional command that can simplify the whole config file, and will +make it more scalable. You have to specify a list of the languages in which +you want to translate the documents. This is as simple as: + + [po4a_langs] fr de + +This will enable you to expand B<$lang> to all the specified languages in the rest +of the config file. + +=head2 Specifying the paths to translator inputs + +B It is recommended to use B<[po_directory]> rather than B<[po4a_langs]> +and B<[po4a_paths]>. +See section B below. + +First, you have to specify where the translator input files (i.e. the files +used by translators to do their job) are located. It can be done by such a line: + + [po4a_paths] doc/l10n/project.doc.pot \ + fr:doc/l10n/fr.po de:doc/l10n/de.po + +The command is thus B<[po4a_paths]>. The first argument is the path to the POT +file to use. All subsequent arguments are of the self-explanatory form: + + : + +If you've defined the template languages, you can rewrite the line above this +way: + + [po4a_paths] doc/l10n/project.doc.pot $lang:doc/l10n/$lang.po + +You can also use B<$master> to refer to the document basename. In this case, +B will use a split mode: one POT and one PO (for each language) will +be created for each document specified in the B configuration file. +See the B section. + + [po4a_paths] doc/$master/$master.pot $lang:doc/$master/$lang.po + +=head2 Autodetection of the paths and languages + +Another command can be used to specify the name of a directory where the +PO and POT files are located. +When it is used, B will detect the POT file as the only F<*.pot> file +from the specified directory. +B will also use the list of F<*.po> files to define the list of +languages (by stripping out the extension). +These languages will be used for the substitution of the B<$lang> variable in +the rest of the configuration file. + +This command should not be used together with the B<[po4a_langs]> or B<[po4a_paths]> +commands. + +When using this command, you have to create an empty POT file on the first +invocation of B to let it know the name of the POT file. + + [po_directory] po4a/po/ + +=head2 Specifying the documents to translate + +You now naturally have to specify which documents are translated, their +format, and where to put the translations. It can be made by such lines: + + [type: sgml] doc/my_stuff.sgml fr:doc/fr/mon_truc.sgml \ + de:doc/de/mein_kram.sgml + [type: pod] script fr:doc/fr/script.1 de:doc/de/script.1 \ + add_fr:doc/l10n/script.fr.add + +This should be rather self-explanatory also. Note that in the second case, +F is an addendum to add to the French version of this document. +Please refer to L for more information about the addenda. + +More formally, the format is: + + [type: ] (:)* \ + (add_:*)* + +If there is no modifier, I is a path to an addendum. +Modifiers are + +=over 2 + +=item B + +Include I if this file does exist, otherwise do nothing. + +=item B<@> + +I is not a regular addendum but a file containg a list of +addenda, one by line. Each addendum may be preceded by modifiers. + +=item B + +I is discarded, it is not loaded and will not be loaded by +any further addendum specification. + +=back + +If you've defined the template languages, you can rewrite the line above this +way: + + [type: pod] script $lang:doc/$lang/script.1 \ + add_fr:doc/l10n/script.fr.add + +If all the languages had addenda with similar paths, you could also write +something like: + + [type: pod] script $lang:doc/$lang/script.1 \ + add_$lang:doc/l10n/script.$lang.add + +=head2 Specifying options for the modules + +B accepts options that will be passed to the module. These options are +module specific and are specified with the B<-o> switch. + +If you need a specific option for one of the document you want to +translate, you can also specify it in the configuration file. Options are +introduced by the B keyword. The argument of the B keyword must be +quoted with double quotes if it contains a space (e.g. if you specify +multiple options, or an option with an argument). +You can also specify options that will only apply to a specific language +by using the BI keyword. + +Here is an example: + [type:man] data-05/test2_man.1 $lang:tmp/test2_man.$lang.1 \ + opt:"-k 75" opt_it:"-L UTF-8" opt_fr:-v + +Arguments may contain spaces if you use single quotes or escaped double +quotes: + [po4a_alias:man] man opt:"-o \"mdoc=NAME,SEE ALSO\" -k 20" + +If you want to specify the same options for many documents, you may want +to use an alias (see the B section below). + +You can also set options for all the documents specified in the +configuration file: + [options] opt:"..." opt_fr:"..." + +=head2 Specifying aliases + +If you must specify the same options for multiple files, you may be +interested in defining a module alias. This can be done this way: + +[po4a_alias:test] man opt:"-k 21" opt_es:"-o debug=splitargs" + +This defines a module alias named B, based on the B module, with +the B<-k 21> applied to all the languages and with B<-o debug=splitargs> +applied to the Spanish translation. + +This module alias can then be use like a regular module: + +[type:test] data-05/test2_man.1 $lang:tmp/test2_man.$lang.1 \ + opt_it:"-L UTF-8" opt_fr:-v + +Note that you can specify additional options on a per file basis. + +=head2 Split mode + +The split mode is used when B<$master> is used in the B<[po4a_paths]> line. + +When the split mode is used, a temporary big POT and temporary big POs +are used. This permits to share the translations between all the POs. + +If two POs have different translations for the same string, B will mark +this string as fuzzy and will submit both translations in all the POs +which contain this string. Then, when a translator updates the translation +and removes the fuzzy tag in one PO, the translation of this string will +be updated in every POs automatically. + +=head1 OPTIONS + +=over 4 + +=item B<-k>, B<--keep> + +Minimal threshold for translation percentage to keep (i.e. write) the +resulting file (default: 80). I.e. by default, files have to be translated +at at least 80% to get written. + +=item B<-h>, B<--help> + +Show a short help message. + +=item B<-M>, B<--master-charset> + +Charset of the files containing the documents to translate. Note that all +master documents must use the same charset for now. This is a known +limitation, and we are working on solving this. + +=item B<-L>, B<--localized-charset> + +Charset of the files containing the localized documents. Note that all +translated documents will use the same charset for now. This is a known +limitation, and we are working on solving this. + +=item B<-A>, B<--addendum-charset> + +Charset of the addenda. Note that all the addenda should be in the same +charset. + +=item B<-V>, B<--version> + +Display the version of the script and exit. + +=item B<-v>, B<--verbose> + +Increase the verbosity of the program. + +=item B<-q>, B<--quiet> + +Decrease the verbosity of the program. + +=item B<-d>, B<--debug> + +Output some debugging information. + +=item B<-o>, B<--option> + +Extra option(s) to pass to the format plugin. Specify each option in the +'IB<=>I' format. See the documentation of each plugin for more +information about the valid options and their meanings. + +=item B<-f>, B<--force> + +Always generate the POT and PO files, even if B considers it is +not necessary. + +The default behavior (when B<--force> is not specified) is the following: + +=over + +If the POT file already exists, it is regenerated if a master document +or the configuration file is more recent. +The POT file is also written in a temporary document and B verifies +that the changes are really needed. + +Also, a translation is regenerated only if its master document, the PO file, +one of its addenda or the configuration file is more recent. +To avoid trying to regenerate translations which do not pass the threshold +test (see B<--keep>), a file with the F<.po4a-stamp> extension can be created +(see B<--stamp>). + +=back + +If a master document includes files, you should use the B<--force> flag +because the modification time of these included files are not taken into +account. + +The PO files are always re-generated based on the POT with B. + +=item B<--stamp> + +Tells B to create stamp files when a translation is not generated +because it does not reach the threshold. These stamp files are named +according to the expected translated document, with the F<.po4a-stamp> +extension. + +Note: This only activates the creation of the F<.po4a-stamp> files. The stamp +files are always used if they exist, and they are removed with +B<--rm-translations> or when the file is finally translated. + +=item B<--no-translations> + +Do not generate the translated documents, only update the POT and PO files. + +=item B<--rm-translations> + +Remove the translated files (implies B<--no-translations>). + +=item B<--no-backups> + +This flag does nothing since 0.41, and may be removed +in later releases. + +=item B<--rm-backups> + +This flag does nothing since 0.41, and may be removed +in later releases. + +=item B<--translate-only> I + +Translate only the specified file. It may be useful to speed up +processing if a configuration file contains a lot of files. Note that this +option does not update PO and POT files. +This option can be used multiple times. + +=item B<--variable> IB<=>I + +Define a variable that will be expanded in the B configuration file. +Every occurrence of I<$(var)> will be replaced by I. +This option can be used multiple times. + +=item B<--msgid-bugs-address> I + +Set the report address for msgid bugs. By default, the created POT files +have no Report-Msgid-Bugs-To fields. + +=item B<--copyright-holder> I + +Set the copyright holder in the POT header. The default value is +"Free Software Foundation, Inc." + +=item B<--package-name> I + +Set the package name for the POT header. The default is "PACKAGE". + +=item B<--package-version> I + +Set the package version for the POT header. The default is "VERSION". + +=item B<--msgmerge-opt> I + +Extra options for B. + +Note: B<$lang> will be extended to the current language. + +=item B<--no-previous> + +This option removes B<--previous> from the options passed to B. +This permits to support versions of B earlier than 0.16. + +=item B<--previous> + +This option adds B<--previous> to the options passed to B. +It requires B 0.16 or later, and is activated by default. + +=item B<--srcdir> I + +Set the base directory for all input documents specified in the B +configuration file. + +=item B<--destdir> I + +Set the base directory for all the output documents specified in the B +configuration file. + +=back + +=head2 EXAMPLE + +Let's assume you maintain a program named B which has a man page F +which naturally is maintained in English only. Now you as the upstream or +downstream maintainer want to create and maintain the translation. +First you need to create the POT file necessary to send to translators +using L. + +So for our case we would call + + cd man && po4a-gettextize -f man -m foo.1 -p foo.pot + +You would then send this file to the appropriate language lists or offer +it for download somewhere on your website. + +Now let's assume you received three translations before your next release: +F (including an addendum F), F and F. +Since you don't want to change your F(s) whenever a new translation +arrives you can use B with an appropriate configuration file in your F. +Let's call it F. In our example it would look like the following: + + [po_directory] man/po4a/po/ + + [type: man] man/foo.1 $lang:man/translated/$lang/foo.1 \ + add_$lang:?man/po4a/add_$lang/$lang.add opt:"-k 80" + +In this example we assume that your generated man pages (and all PO and addenda +files) should be stored in F (respectively in F and +F) below the current directory. In our example +the F directory would include F, F and F, +and the F directory would include F. + +Note the use of the modifier B as only the German translation (F) is +accompanied by an addendum. + +To actually build the translated man pages you would then (once!) add the +following line in the B target of the appropriate F: + + po4a po4a.cfg + +Once this is set up you don't need to touch the F when a new +translation arrives, i.e. if the French team sends you F and F +then you simply drop them respectively in F and +F and the next time the programm is build the +French translation is automatically build as well in F. + +Note that you still need an appropriate target to install localized manual +pages with English ones. + +Finally if you do not store generated files into your version control system, +you will need a line in your B target as well: + -rm -rf man/translated + +=head1 SHORTCOMINGS + +=over 4 + +=item + +Duplicates some code with the BI<*> programs. + +=back + +Patch welcome ;) + +=head1 SEE ALSO + +L, +L, +L, +L, +L, +L, +L + +=head1 AUTHORS + + Denis Barbier + Nicolas François + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002-2012 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +use 5.006; +use strict; +use warnings; + +use Getopt::Long qw(GetOptions); + +use Locale::Po4a::Chooser; +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; +use Locale::Po4a::Po qw(move_po_if_needed); + +use Pod::Usage qw(pod2usage); + +use File::Temp; +use File::Basename; +use File::Copy; +use File::Spec; +use Fcntl; # sysopen flags +use Cwd; # cwd + +Locale::Po4a::Common::textdomain('po4a'); + + +sub show_version { + Locale::Po4a::Common::show_version("po4a"); + exit 0; +} + +# keep the command line arguments +my @ORIGINAL_ARGV = @ARGV; + +# Parse the options provided on the command line, or in argument +sub get_options { + if (defined $_[0]) { + @ARGV = @_; + } else { + @ARGV = @ORIGINAL_ARGV; + } + + # temporary array for GetOptions + my @verbose = (); + my @options = (); + my @variables = (); + my $previous; + my $noprevious; + + my %opts = ( + "help" => 0, + "type" => "", + "debug" => 0, + "verbose" => 0, + "quiet" => 0, + "no-translations" => 0, + "rm-translations" => 0, + "no-backups" => 0, + "rm-backups" => 0, + "threshold" => 80, + "mastchar" => "", + "locchar" => "", + "addchar" => "", + "options" => {"verbose" => 0, "debug" => 0}, + "variables" => {}, + "partial" => [], + "copyright-holder"=> undef, + "msgid-bugs-address"=> undef, + "package-name" => undef, + "package-version" => undef, + "msgmerge-opt" => "", + "srcdir" => undef, + "destdir" => undef, + "calldir" => cwd() + ); + Getopt::Long::config('bundling', 'no_getopt_compat', 'no_auto_abbrev'); + GetOptions( + 'help|h' => \$opts{"help"}, + + 'master-charset|M=s' => \$opts{"mastchar"}, + 'localized-charset|L=s' => \$opts{"locchar"}, + 'addendum-charset|A=s' => \$opts{"addchar"}, + + 'verbose|v' => \@verbose, + 'debug|d' => \$opts{"debug"}, + 'force|f' => \$opts{"force"}, + 'stamp' => \$opts{"stamp"}, + 'quiet|q' => \$opts{"quiet"}, + 'keep|k=s' => \$opts{"threshold"}, + 'no-translations' => \$opts{"no-translations"}, + 'rm-translations' => \$opts{"rm-translations"}, + 'translate-only=s' => \@{$opts{"partial"}}, + 'no-backups' => \$opts{"no-backups"}, + 'rm-backups' => \$opts{"rm-backups"}, + 'version|V' => \&show_version, + 'option|o=s' => \@options, + 'variable=s' => \@variables, + 'copyright-holder=s' => \$opts{"copyright-holder"}, + 'msgid-bugs-address=s' => \$opts{"msgid-bugs-address"}, + 'package-name=s' => \$opts{"package-name"}, + 'package-version=s' => \$opts{"package-version"}, + 'no-previous' => \$noprevious, + 'previous' => \$previous, + 'msgmerge-opt=s' => \$opts{"msgmerge-opt"}, + 'srcdir=s' => \$opts{"srcdir"}, + 'destdir=s' => \$opts{"destdir"} + ) or pod2usage(); + + $opts{"verbose"} = scalar @verbose; + $opts{"verbose"} = 0 if $opts{"quiet"}; + $opts{"verbose"} ||= 1 if $opts{"debug"}; + $opts{"msgmerge-opt"} .= " --previous" unless $noprevious; + + # options to transmit to the modules + $opts{"options"} = { + "verbose" => $opts{"verbose"}, + "debug" => $opts{"debug"} + }; + foreach (@options) { + if (m/^([^=]*)=(.*)$/) { + $opts{"options"}{$1}="$2"; + } else { + $opts{"options"}{$_}=1; + } + } + + foreach (@variables) { + if (m/^([^=]*)=(.*)$/) { + $opts{"variables"}{$1}="$2"; + } + } + + # The rm- options imply the no- + $opts{"no-translations"} = 1 if $opts{"rm-translations"}; + + if (defined $opts{"srcdir"} and not -d $opts{"srcdir"}) { + die wrap_msg(gettext("Invalid %s. Directory %s does not exist."), + "srcdir", $opts{"srcdir"}); + } + if (defined $opts{"destdir"} and not -d $opts{"destdir"}) { + die wrap_msg(gettext("Invalid %s. Directory %s does not exist."), + "destdir", $opts{"destdir"}); + } + + return %opts; +} + +# Parse a config line and extract the parameters that correspond to options. +# These options are appended to the options provided in argument (as a +# reference to an hash). The options are sorted by category in this hash. +# The categories are: global and the various languages. +sub parse_config_options { + my $ref = shift; # a line reference for the die messages + my $line = shift; # the line to parse + my $orig_line = $line; # keep the original line for die messages + my $options = shift; # reference to an hash of options + + while (defined $line and $line !~ m/^\s*$/) { + if ($line =~ m/^\s*opt(?:_(.+?))?:\s*(.+?)\s*$/) { + my $lang = $1; + $line = $2; + my $opt = ""; + if ($line =~ m/^\s*"(.+?(?{$lang}) { + $options->{$lang} = $opt; + } else { + $options->{$lang} .= " $opt"; + } + } else { + last; + } + } + + $line = "" unless defined $line; + return $line; +} + +my %po4a_opts = get_options(@ARGV); +# Argument check +$po4a_opts{"help"} && pod2usage (-verbose => 1, -exitval => 0); + +sub run_cmd { + my $cmd = shift; + print $cmd."\n" if $po4a_opts{"debug"}; + + my $out = qx/$cmd 2>&1/; + print $out if ($po4a_opts{"verbose"}); + unless ($? == 0) { + my $err = ""; + if ($? == -1) { + $err = sprintf(gettext("failed to execute '%s': %s."), + $cmd, $!); + } elsif ($? & 127) { + if ($? & 128) { + $err = sprintf(gettext("'%s' died with signal %d, ". + "with coredump."), + $cmd, $? & 127); + } else { + $err = sprintf(gettext("'%s' died with signal %d, ". + "without coredump."), + $cmd, $? & 127); + } + } else { + $err = sprintf(gettext("'%s' exited with value %d."), + $cmd, $? >> 8); + } + + die wrap_msg(gettext("Error: %s"), $err); + } +} + +my $config_file= shift(@ARGV) || pod2usage(); +# Check file existence +-e $config_file || die wrap_msg(gettext("File %s does not exist."), $config_file); + +# Parse the config file +my (@langs); +my (%aliases); # module aliases ([po4a_alias:...] +my ($pot_filename) = ""; +my (%po_filename); # po_files: '$lang'=>'$path' +my (%document); # '$master'=> {'format'=>'$format'; '$lang'=>'$path'; 'add_$lang'=>('$path','$path') } +my $doc_count = 0; +my %partial = ( 'master' => {}, 'files' => {}, 'lang' => {} ); +open CONFIG,"<","$config_file" or die wrap_msg(gettext("Can't open %s: %s"), $config_file, $!); +my ($line,$nb) = ("",0); +while () { + $nb++; + s/#.*//; + $line.=$_; + $line =~ s/\t/ /g; + $line =~ s/ +/ /g; + while ($line =~ m/\$\((\w+)\)/) { + if (defined $po4a_opts{"variables"}{$1}) { + $line =~ s/\$\((\Q$1\E)\)/$po4a_opts{"variables"}{$1}/g; + } else { + die wrap_ref_mod("$config_file:$nb", "", + gettext("Unknown variable: %s"), $1); + } + } + $line =~ s/^ //; + $line =~ s/ $//; + chomp($line); + next if ($line =~ s/\\$//); + next unless ($line =~ /\S/); + + my $args = $line; + die wrap_ref_mod("$config_file:$nb", "", gettext("Syntax error: %s"), $line) + unless ($args =~ s/^\[([^\]]*)\] *//); + my $cmd = $1; + my $main = ($args=~ s/^(\S+) *// ? $1 : ""); + + if (@langs) { + # Expand the $lang templates + my($args2) = ""; + foreach my $arg (split(/ /,$args)) { + if ( $arg =~ /\$lang\b/ ) { + # Expand for all the langs + foreach my $lang (@langs) { + my($arg2) = $arg; + $arg2 =~ s/\$lang\b/$lang/g; + $args2 .= $arg2." "; + } + } else { + # Leave the argument as is + $args2 .= $arg." "; + } + } + $args = $args2; + } + + print "cmd=[$cmd]; main=$main; args=\"$args\"\n" if $po4a_opts{"debug"}; + + if ($cmd eq "po4a_paths") { + die wrap_ref_mod("$config_file:$nb", "", + gettext("'%s' redeclared"), "po4a_path") + if (length $pot_filename); + $pot_filename = $main; + foreach my $arg (split(/ /,$args)) { + die wrap_ref_mod("$config_file:$nb", "", + gettext("Unparsable argument '%s'."), $arg) + unless ($arg =~ /^([^:]*):(.*)/); + $po_filename{$1}=$2 if $1 ne '$lang'; + } + + } elsif ($cmd eq "po4a_langs") { + die wrap_ref_mod("$config_file:$nb", "", + gettext("'%s' redeclared"), "po4a_langs") + if (@langs); + @langs = split(/ /,$main." ".$args); + + } elsif ($cmd eq "po_directory") { + die wrap_ref_mod("$config_file:$nb", "", + gettext("The list of languages cannot be set twice.")) + if scalar @langs; + die wrap_ref_mod("$config_file:$nb", "", + gettext("The POT file cannot be set twice.")) + if length $pot_filename; + + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + my $po_directory = $main; + die wrap_ref_mod("$config_file:$nb", "", + gettext("'%s' is not a directory"), $po_directory) + unless (-d $po_directory); + opendir PO_DIR, $po_directory + or die wrap_ref_mod("$config_file:$nb", "", + gettext("Cannot list the '%s' directory"), $po_directory); + + foreach my $f (readdir PO_DIR) { + next unless -f "$po_directory/$f"; + if ($f =~ m/^(.*)\.po$/) { + push @langs, $1; + $po_filename{$1} = "$po_directory/$f"; + } + if ($f =~ m/\.pot$/) { + if (length $pot_filename) { + die wrap_ref_mod("$config_file:$nb", "", + gettext("too many POT files: %s %s"), + $pot_filename, "$po_directory/$f"); + } else { + $pot_filename = "$po_directory/$f"; + } + } + } + + if (not @langs) { + warn wrap_ref_mod("$config_file:$nb", "", + gettext("no PO files found in %s"), $po_directory); + } + + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + } elsif ($cmd =~ m/type: *(.*)/) { + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + + if (defined $document{$main}{'format'}) { + warn wrap_ref_mod("$config_file:$nb", "", + gettext("The '%s' master file was specified earlier in the ". + "configuration file. This may cause problems with ". + "options."), $main) + unless ($po4a_opts{"quiet"}); + } elsif (not -e $main) { + die wrap_ref_mod("$config_file:$nb", "", + gettext("The '%s' master file does not exist."), $main); + } + if (scalar @{$po4a_opts{"partial"}}) { + foreach my $file (@{$po4a_opts{"partial"}}) { + if ($args =~ m/(\S+):\Q$file\E\b/) { + $partial{'lang'}{$1} = 1; + $partial{'master'}{$main} = 1; + $partial{'files'}{$file} = 1; + last; + } + } + } + $document{$main}{'format'} = $1; + $document{$main}{'pos'} = $doc_count; + $doc_count++; + + # Options + my %options; + # 1. Use the global options ([opt] ...) + %options = %{$document{''}{'options'}} + if defined $document{''}{'options'}; + + # 2. Merge the alias options + if (defined $aliases{$1}) { + $document{$main}{'format'} = $aliases{$1}{"module"}; + if (defined $aliases{$1}{"options"}) { + %options = %{$aliases{$1}{"options"}}; # XXX not a merge, but overwrite + } + } + + # 3. If this file was already specified, reuse the previous + # options (no merge) + %options = %{$document{$main}{'options'}} + if defined $document{$main}{'options'}; + + # 4. Merge the document specific options + # separate the end of the line, which contains options. + # Something more clever could be done to allow options in the + # middle of a line. + if ($args =~ m/^(.*?) +(opt(_.+)?:(.*))$/) { + $args = $1; + $args = "" unless defined $args; + $args .= " ".parse_config_options("$config_file:$nb", + $2, \%options); + } + %{$document{$main}{'options'}} = %options; + + my %discarded = (); + my @remaining_args = split(/ /,$args); + while (@remaining_args) { + my $arg = shift(@remaining_args); + die wrap_ref_mod("$config_file:$nb", "", + gettext("Unparsable argument '%s' (%s)."), $arg, $line) + unless ($arg =~ /^([^:]*):(.*)/); + my ($lang,$trans)=($1,$2); + die wrap_ref_mod("$config_file:$nb", "", + gettext("The translated and master file are the same.")) + if ($main eq $trans); + + if ($lang =~ /^add_/) { + my $modifiers; + $trans =~ s/^([@!?]+)// and $modifiers = $1; + next if defined($discarded{$trans}); + if (defined $modifiers) { + if ($modifiers =~ m/!/) { + $discarded{$trans} = 1; + next; + } + next if ($modifiers =~ m/\?/ and not -e $trans); + if ($modifiers =~ m/@/) { + open LIST,"<","$trans" or die wrap_msg(gettext("Can't open %s: %s"), $trans, $!); + my @new_list = (); + while() { + chomp; + s/\s+$//; + next if length($_) == 0 or $_ =~ m/^\s*#/; + while (m/\$\((\w+)\)/) { + if (defined $po4a_opts{"variables"}{$1}) { + s/\$\((\Q$1\E)\)/$po4a_opts{"variables"}{$1}/g; + } else { + die wrap_ref_mod("$config_file:$nb", "", + gettext("Unknown variable: %s"), $1); + } + } + push(@new_list, "$lang:$_"); + } + close LIST; + unshift(@remaining_args, @new_list); + } else { + push @{$document{$main}{$lang}},$trans; + } + } else { + push @{$document{$main}{$lang}},$trans; + } + } else { + die wrap_ref_mod("$config_file:$nb", "", + gettext("Translation of %s in %s redefined"), $main, $lang) + if (defined $document{$main}{$lang}); + $document{$main}{$lang} = $trans; + } + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + } elsif ($cmd =~ m/po4a_alias: *(.*)/) { + my $name = $1; + my %alias = (); + $alias{"module"} = $main; + my %options; + $args = parse_config_options("$config_file:$nb", $args, \%options); + %{$alias{"options"}} = %options; + %{$aliases{$name}} = %alias; + } elsif ($cmd eq "options") { + my %options; + my $o = $line; + $o =~ s/.*?\[options\] +//; + if (defined $document{''}{"options"}) { + %options = %{$document{''}{"options"}}; + } + parse_config_options("$config_file:$nb", + $o, + \%options); + %{$document{''}{"options"}} = %options; + } else { + die wrap_ref_mod("$config_file:$nb", "", + gettext("Unparsable command '%s'."), $cmd); + } + + $line = ""; +} +close CONFIG; # don't care about error here +die wrap_msg(gettext("po4a_paths not declared. Dunno where to find the POT and PO files")) + unless (length $pot_filename); + +sub split_opts { + my $options = shift; + my $options_orig = $options; + my @opts = (); + $options =~ s/\\"/"/g; + while (length $options) { + my $o = ""; + while (length $options and $options !~ /^ /s) { + if ( ($options =~ m/^(["'])/) + and ($options !~ m/^(["'])(?:\\.|(?!\1)[^\\])*\1/)) { + die wrap_msg(gettext("Cannot parse option line (missing >%s '$path' +my %split_pot; # pot_files: '$master' => '$path' + +# make a big pot +my $update_pot_file = 0; +if ($pot_filename =~ m/\$master/) { + print wrap_msg(gettext("Split mode, creating a temporary POT")."\n") + if $po4a_opts{"verbose"}; + if (scalar @{$po4a_opts{"partial"}}) { + print wrap_msg(gettext("Disabling --translate-only option, it is not supported in split mode")."\n"); + $po4a_opts{"partial"} = []; + } + foreach my $master (keys %document) { + next if ($master eq ''); + my $m = basename $master; + my $master_pot = $pot_filename; + $master_pot =~ s/\$master/$m/g; + $split_pot{$master} = $master_pot; + } + # The POT needs to be generated anyway. + $update_pot_file = 1; + $po4a_opts{"split"} = 1; +} else { + if (scalar @{$po4a_opts{"partial"}}) { + # Skip documents not specified, strings are read directly from POT file + foreach my $master (keys %document) { + next unless length $master; + delete $document{$master} unless exists $partial{'master'}{$master}; + } + # Do not read PO files if no file is processed for this language + foreach my $lang (keys %po_filename) { + delete $po_filename{$lang} unless exists $partial{'lang'}{$lang}; + } + } + +if (not scalar @{$po4a_opts{"partial"}}) { +chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); +if (-e $pot_filename) { + my $modtime = (stat $pot_filename)[9]; + # The POT needs to be re-generated if a master document is more recent + # than the POT. + foreach my $master (keys %document) { + next if ($master eq ''); + if ((stat $master)[9] >= $modtime) { + $update_pot_file = 1; + last; + } + } + + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + if ((stat $config_file)[9] > $modtime) { + # The configuration file was modified after the POT. + # Maybe a new document, or new options + $update_pot_file = 1; + } + + if ($po4a_opts{"force"}) { + $update_pot_file = 1; + } + + print wrap_msg(gettext("Updating %s:"), $pot_filename) + if ($update_pot_file and $po4a_opts{"verbose"}); +} else { + print wrap_msg(gettext("Creating %s:"), $pot_filename) + if $po4a_opts{"verbose"}; + $update_pot_file = 1; +} +chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); +} +} + +my %Po_opts; +if (defined $po4a_opts{'msgid-bugs-address'}) { + $Po_opts{'msgid-bugs-address'} = $po4a_opts{'msgid-bugs-address'}; +} +if (defined $po4a_opts{'copyright-holder'}) { + $Po_opts{'copyright-holder'} = $po4a_opts{'copyright-holder'}; +} +if (defined $po4a_opts{'package-name'}) { + $Po_opts{'package-name'} = $po4a_opts{'package-name'}; +} +if (defined $po4a_opts{'package-version'}) { + $Po_opts{'package-version'} = $po4a_opts{'package-version'}; +} +if ($update_pot_file) { + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + my $potfile=Locale::Po4a::Po->new(\%Po_opts); + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + foreach my $master (sort { return -1 if ($a eq ""); + return 1 if ($b eq ""); + $document{$a}{'pos'} <=> $document{$b}{'pos'} + } keys %document) { + next if ($master eq ''); + my %file_opts = %po4a_opts; + my $options = $document{$master}{"options"}{"global"}; + if (defined $options) { + %file_opts = get_options(@ORIGINAL_ARGV, + split_opts($options)); + } + my $doc=Locale::Po4a::Chooser::new($document{$master}{'format'}, + %{$file_opts{"options"}}); + + + # We ensure that the generated po will be in utf-8 if the input document + # isn't entirely in ascii + $doc->{TT}{utf_mode} = 1; + + $doc->setpoout($potfile); + my @file_in_name; + push @file_in_name, $master; + $doc->process('file_in_name' => \@file_in_name, + 'file_in_charset' => $file_opts{"mastchar"}, + 'srcdir' => $po4a_opts{"srcdir"}, + 'destdir' => $po4a_opts{"destdir"}, + 'calldir' => $po4a_opts{"calldir"}); + $potfile = $doc->getpoout(); + } + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + if ($po4a_opts{"split"}) { + (undef,$pot_filename)=File::Temp->tempfile("po4aXXXX", + DIR => "/tmp", + SUFFIX => ".pot", + OPEN => 0, + UNLINK => 0) + or die wrap_msg(gettext("Can't create a temporary POT file: %s"), + $!); + $potfile->write($pot_filename); + } else { + if ($po4a_opts{"force"}) { + $potfile->write($pot_filename); + } else { + $potfile->write_if_needed($pot_filename); + } + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + + print wrap_msg(gettext(" (%d entries)"), $potfile->count_entries()) + unless ($po4a_opts{"quiet"}); +} + +if ($po4a_opts{"split"}) { + # Generate a .pot for each document + foreach my $master (keys %document) { + next if ($master eq ''); + my $tmp_file; + # Create a temporary POT, and check if the old one needs to be + # updated (unless --force was specified). + unless ($po4a_opts{"force"}) { + (undef,$tmp_file)=File::Temp->tempfile("po4aXXXX", + DIR => "/tmp", + SUFFIX => ".pot", + OPEN => 0, + UNLINK => 0) + or die wrap_msg(gettext("Can't create a temporary POT file: %s"), + $!); + } + + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + my $dir = dirname($split_pot{$master}); + if (not -d $dir) { + mkdir $dir + or die wrap_msg(gettext("Can't create directory '%s': %s"), + $dir, $!); + } + my $cmd = "msggrep -N '$master' -o ". + ($po4a_opts{"force"}?$split_pot{$master}:$tmp_file). + " $pot_filename"; + run_cmd($cmd); + + unless ($po4a_opts{"force"}) { + move_po_if_needed($tmp_file, $split_pot{$master}, 0); + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + } + # Generate a complete .po + foreach my $lang (sort keys %po_filename) { + my $tmp_bigpo; + (undef,$tmp_bigpo)=File::Temp->tempfile("po4aXXXX", + DIR => "/tmp", + SUFFIX => "-$lang.po", + OPEN => 0, + UNLINK => 0) + or die wrap_msg(gettext("Can't create a temporary PO file: %s"), + $!); + my $cmd_cat = ""; + foreach my $master (keys %document) { + next if ($master eq ''); + my $m = basename $master; + my $master_po = $po_filename{$lang}; + $master_po =~ s/\$master/$m/g; + if (-e "$master_po") { + $cmd_cat .= " $master_po"; + } + $split_po{$lang}{$master} = $master_po; + } + if (length $cmd_cat) { + $cmd_cat = "msgcat -o $tmp_bigpo $cmd_cat"; + run_cmd($cmd_cat); + } + # We do not need to keep the original name with $master + $po_filename{$lang} = $tmp_bigpo; + } +} + +# update all po files +my $lang; +if (not scalar @{$po4a_opts{"partial"}}) { +foreach $lang (sort keys %po_filename) { + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + if (-e $po_filename{$lang}) { + print wrap_msg(gettext("Updating %s:")." ", $po_filename{$lang}) + if ($po4a_opts{"verbose"}); + my $msgmerge_opt = $po4a_opts{"msgmerge-opt"}; + $msgmerge_opt =~ s/\$lang\b/$lang/g if scalar @langs; + my $cmd = "msgmerge -U ".$po_filename{$lang}." $pot_filename ".$msgmerge_opt." --backup=none"; + run_cmd($cmd); + system "msgfmt --statistics -v -o /dev/null ".$po_filename{$lang} + if $po4a_opts{"verbose"}; + } else { + print wrap_msg(gettext("Creating %s:"), $po_filename{$lang}) + if $po4a_opts{"verbose"}; + my $cmd = "msginit -i $pot_filename --locale $lang -o ".$po_filename{$lang}." --no-translator"; + run_cmd($cmd); + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); +} +} + +if ($po4a_opts{"split"}) { + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + # We don't need the tmp big POT anymore + unlink($pot_filename); + + # Split the complete PO in multiple POs + foreach $lang (sort keys %po_filename) { + foreach my $master (keys %document) { + next if ($master eq ''); + my $tmp_file; + # Create a temporary PO, and check if the old one needs to be + # updated (unless --force was specified). + (undef,$tmp_file)=File::Temp->tempfile("po4aXXXX", + DIR => "/tmp", + SUFFIX => ".po", + OPEN => 0, + UNLINK => 0) + or die wrap_msg( + gettext("Can't create a temporary POT file: %s"), $!); + + my $cmd; + # Create an empty PO or copy the original PO header. + # This permits to keep the header. + if (-f $split_po{$lang}{$master}) { + $cmd = "msggrep --force-po -v -K -e '.'". + " -o ".$tmp_file. + " ".$split_po{$lang}{$master}; + } else { + $cmd = "msginit --no-translator -l ".$lang. + " -i ".$split_pot{$master}. + " -o ".$tmp_file; + } + run_cmd($cmd); + + # Update the PO according to the new POT and to the big PO + # (compendium). + $cmd = "msgmerge -U -C ".$po_filename{$lang}. + " --backup=none ".$po4a_opts{"msgmerge-opt"}. + " $tmp_file ".$split_pot{$master}; + run_cmd($cmd); + + my $dir = dirname($split_po{$lang}{$master}); + if (not -d $dir) { + mkdir $dir + or die wrap_msg(gettext("Can't create directory '%s': %s"), + $dir, $!); + } + unless ($po4a_opts{"force"}) { + move_po_if_needed($tmp_file, + $split_po{$lang}{$master}, + 0); + } else { + move $tmp_file, $split_po{$lang}{$master} + or die wrap_msg(dgettext("po4a", + "Can't move %s to %s: %s."), + $tmp_file, + $split_po{$lang}{$master}, + $!); + } + } + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); +} + +if (not $po4a_opts{"no-translations"}) { + # update all translations + + foreach $lang (sort keys %po_filename) { + # Read the $lang PO once, no options for the creation + my $po = Locale::Po4a::Po->new(); + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + $po->read($po_filename{$lang}); + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + + DOC: foreach my $master (sort { return -1 if ($a eq ""); + return 1 if ($b eq ""); + $document{$a}{'pos'} <=> + $document{$b}{'pos'} } keys %document) { + next if ($master eq ''); + next unless defined $document{$master}{$lang}; + if (scalar @{$po4a_opts{"partial"}}) { + next unless defined $partial{'files'}{$document{$master}{$lang}}; + } + + unless ($po4a_opts{"force"}) { + chdir $po4a_opts{"destdir"} + if (defined $po4a_opts{"destdir"}); + my $stampfile = $document{$master}{$lang}; + unless (-e $document{$master}{$lang}) { + $stampfile = $document{$master}{$lang}.".po4a-stamp"; + } + $stampfile = File::Spec->rel2abs($stampfile); + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"destdir"}); + + my @files = ($master, + $po_filename{$lang}, + File::Spec->rel2abs($config_file)); + if (defined $document{$master}{"add_$lang"}) { + push @files, @{$document{$master}{"add_$lang"}}; + } + + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + unless (is_older($stampfile, @files)) { + print wrap_msg(gettext("%s doesn't need to be updated."), + $document{$master}{$lang}) + if ($po4a_opts{"verbose"}); + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + next DOC; + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + } + + my %file_opts = %po4a_opts; + my $options = ""; + if (defined $document{$master}{"options"}{"global"}) { + $options .= $document{$master}{"options"}{"global"}; + } + # append the language options + if (defined $document{$master}{"options"}{$lang}) { + $options .= " ".$document{$master}{"options"}{$lang}; + } + if (defined $options) { + # also use the options provided on the command line + %file_opts = get_options(@ORIGINAL_ARGV, + split_opts($options)); + } + my $doc=Locale::Po4a::Chooser::new($document{$master}{'format'}, + %{$file_opts{"options"}}); + + my @file_in_name; + push @file_in_name, $master; + + # Reuse the already parsed PO, do not use the po_in_name + # option of process. + $doc->{TT}{po_in} = $po; + $doc->{TT}{po_in}->stats_clear(); + + $doc->process('file_in_name' => \@file_in_name, + 'file_out_name' => $document{$master}{$lang}, + 'file_in_charset' => $file_opts{"mastchar"}, + 'file_out_charset' => $file_opts{"locchar"}, + 'addendum_charset' => $file_opts{"addchar"}, + 'srcdir' => $po4a_opts{"srcdir"}, + 'destdir' => $po4a_opts{"destdir"}, + 'calldir' => $po4a_opts{"calldir"}); + + my ($percent,$hit,$queries) = $doc->stats(); + + if ($percent<$file_opts{"threshold"}) { + print wrap_msg(gettext("Discard %s (%s of %s strings; only %s%% translated; need %s%%)."), + $document{$master}{$lang}, $hit, $queries, + $percent, $file_opts{"threshold"}); + chdir $po4a_opts{"destdir"} + if (defined $po4a_opts{"destdir"}); + unlink($document{$master}{$lang}) if (-e $document{$master}{$lang}); + unless ($po4a_opts{"force"}) { + if ($po4a_opts{"stamp"}) { + touch($document{$master}{$lang}.".po4a-stamp"); + print wrap_msg(gettext("Timestamp %s created."), + $document{$master}{$lang}.".po4a-stamp") + if ($po4a_opts{"verbose"}); + } + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"destdir"}); + next DOC; + } + unless ($po4a_opts{"force"}) { + chdir $po4a_opts{"destdir"} + if (defined $po4a_opts{"destdir"}); + if (-e $document{$master}{$lang}.".po4a-stamp") { + unlink $document{$master}{$lang}.".po4a-stamp"; + print wrap_msg(gettext("Timestamp %s removed."), + $document{$master}{$lang}.".po4a-stamp") + if ($po4a_opts{"verbose"}); + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"destdir"}); + } + + if (defined ($document{$master}{"add_$lang"})) { + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + foreach my $add (@{$document{$master}{"add_$lang"}}) { + if ( !$doc->addendum($add) ) { + die wrap_msg(gettext("Addendum %s does NOT apply to %s (translation discarded)."), + $add, $document{$master}{$lang}); + chdir $po4a_opts{"destdir"} + if (defined $po4a_opts{"destdir"}); + unlink($document{$master}{$lang}) if (-e $document{$master}{$lang}); + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"destdir"}); + next DOC; + } + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); + } + if ($file_opts{"verbose"}) { + if ($percent == 100) { + print wrap_msg(gettext("%s is %s%% translated (%s strings)."), + $document{$master}{$lang}, $percent, $queries); + } else { + print wrap_msg(gettext("%s is %s%% translated (%s of %s strings)."), + $document{$master}{$lang}, $percent, $hit, $queries); + } + } + + chdir $po4a_opts{"destdir"} + if (defined $po4a_opts{"destdir"}); + $doc->write($document{$master}{$lang}); + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"destdir"}); + } + } +} + +if ($po4a_opts{"split"}) { + chdir $po4a_opts{"srcdir"} + if (defined $po4a_opts{"srcdir"}); + # We don't need the tmp big POs anymore + foreach $lang (keys %po_filename) { + unlink $po_filename{$lang}; + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"srcdir"}); +} + +if ($po4a_opts{"rm-translations"}) { + # Delete the translated documents + chdir $po4a_opts{"destdir"} + if (defined $po4a_opts{"destdir"}); + foreach $lang (keys %po_filename) { + foreach my $master (keys %document) { + next if ($master eq ''); + unlink $document{$master}{$lang}; + unlink $document{$master}{$lang}.".po4a-stamp"; + } + } + chdir $po4a_opts{"calldir"} + if (defined $po4a_opts{"destdir"}); +} + +sub touch { + my $file = shift; + if (-e $file) { + utime undef, undef, $file; + } else { + sysopen(FH,$file,O_WRONLY|O_CREAT|O_NONBLOCK|O_NOCTTY) + or croak("Can't create $file : $!"); + close FH or croak("Can't close $file : $!"); + } +} + +sub is_older { + my $file = shift; + my @files = @_; + return 1 unless (-e $file); + my $older = 0; + + my $modtime = (stat $file)[9]; + + for my $f (@files) { + if (-e $f and (stat $f)[9] > $modtime) { + $older = 1; + last; + } + } + + return $older; +} + +__END__ Property changes on: branches/Bungo/package/bin/po4a/po4a ___________________________________________________________________ Added: svn:executable + * Index: branches/Bungo/package/bin/po4a/MANIFEST =================================================================== --- branches/Bungo/package/bin/po4a/MANIFEST (revision 0) +++ branches/Bungo/package/bin/po4a/MANIFEST (revision 2840) @@ -0,0 +1,333 @@ +MANIFEST This list of files (needed by ModuleBuild) +META.yml Module meta-data (added by ModuleBuild) +COPYING +changelog +Build.PL File processed to create the Build script +Po4aBuilder.pm The rules to build this module (for ModuleBuild) +README +README.maintainers +README.tests +README.translators +TODO +NEWS + +doc/addendum.ca +doc/addendum.es +doc/addendum.fr +doc/addendum.it +doc/addendum.ja +doc/addendum.pl +doc/addendum_man.es +doc/addendum_man.fr +doc/addendum_man.ja +doc/addendum_man.pl +doc/po4a.7.pod +doc/po4a-build.conf.5.pod +doc/po4a-runtime.7.pod +lib/Locale/Po4a/BibTeX.pm +lib/Locale/Po4a/Chooser.pm +lib/Locale/Po4a/Common.pm +lib/Locale/Po4a/Dia.pm +lib/Locale/Po4a/Docbook.pm +lib/Locale/Po4a/Guide.pm +lib/Locale/Po4a/Halibut.pm +lib/Locale/Po4a/Ini.pm +lib/Locale/Po4a/KernelHelp.pm +lib/Locale/Po4a/LaTeX.pm +lib/Locale/Po4a/Man.pm +lib/Locale/Po4a/Po.pm +lib/Locale/Po4a/Pod.pm +lib/Locale/Po4a/Sgml.pm +lib/Locale/Po4a/TeX.pm +lib/Locale/Po4a/Texinfo.pm +lib/Locale/Po4a/Text.pm +lib/Locale/Po4a/TransTractor.pm +lib/Locale/Po4a/Xhtml.pm +lib/Locale/Po4a/Xml.pm +lib/Locale/Po4a/Wml.pm + +po/bin/af.po +po/bin/bn.po +po/bin/ca.po +po/bin/cs.po +po/bin/da.po +po/bin/de.po +po/bin/eo.po +po/bin/es.po +po/bin/et.po +po/bin/eu.po +po/bin/fr.po +po/bin/he.po +po/bin/hr.po +po/bin/id.po +po/bin/ja.po +po/bin/it.po +po/bin/kn.po +po/bin/ko.po +po/bin/ku.po +po/bin/nb.po +po/bin/nl.po +po/bin/oc.po +po/bin/pl.po +po/bin/po4a.pot +po/bin/pt_BR.po +po/bin/pt.po +po/bin/ru.po +po/bin/sl.po +po/bin/sv.po +po/bin/uk.po +po/bin/uz.po +po/bin/vi.po +po/bin/zh_CN.po +po/bin/zh_HK.po +po/bin/POTFILES.in +po/bin/Makevars +po/bin/Makefile +po/bin/LINGUAS +po/pod/ca.po +po/pod/de.po +po/pod/es.po +po/pod/fr.po +po/pod/it.po +po/pod/ja.po +po/pod/pl.po +po/pod/po4a-pod.pot +po/pod/ru.po +po/pod.cfg + +# the binaries +po4a +po4a-gettextize +po4a-normalize +po4a-translate +po4a-updatepo +po4a-build.conf +share/po4a-build + +# support files +share/Makefile +share/Makevars-perl.example +share/Makevars-shell.example +share/po4a-build.make +share/README +share/po4a-build.conf.example +share/doc/po4a-build.xml +share/doc/po4aman-display-po.xml +share/doc/po4apod-display-po.xml + +# the random additional cool scripts +scripts/msguntypot +scripts/po4aman-display-po +scripts/po4apod-display-po + +# The tests +t/compare-po.pl + +t/01-classes.t + +t/02-TransTractors.t +t/data-02/man +t/data-02/man.fr +t/data-02/man.fr-normalized +t/data-02/man.po +t/data-02/man.po-empty +t/data-02/man.po-ok +t/data-02/pod +t/data-02/pod.fr +t/data-02/pod.fr-normalized +t/data-02/pod.po +t/data-02/pod.po-empty +t/data-02/pod.po-ok + +t/03-addendums.t +t/data-03/man +t/data-03/man.addendum1 +t/data-03/man.addendum2 +t/data-03/man.addendum3 +t/data-03/man.addendum4 +t/data-03/man.fr +t/data-03/man.fr.add1 +t/data-03/man.fr.add2 +t/data-03/man.fr.add3 +t/data-03/man.fr.add4 +t/data-03/man.po-ok + +t/04-charsets.t +t/data-04/ascii.po-ok +t/data-04/ascii-iso8859.po-ok +t/data-04/iso8859.po-ok +t/data-04/text-ascii.pod +t/data-04/text-iso8859.pod +t/data-04/text-iso8859_.pod +t/data-04/text-iso8859.pod-ok +t/data-04/trans.po +t/data-04/utf.po +t/data-04/utf.po-ok +t/data-04/utf.pod-ok + +t/10-discard-fuzzy.t +t/data-10/pod +t/data-10/pod.fr +t/data-10/pod.po + +t/11-plural.t +t/data-11/err1 +t/data-11/pod1 +t/data-11/pod1.fr +t/data-11/pod1.po +t/data-11/pod2 +t/data-11/pod2.fr +t/data-11/pod2.po +t/data-11/pod3 +t/data-11/pod3.fr +t/data-11/pod4 +t/data-11/pod4.fr + +t/20-sgml.t +t/data-20/text.xml +t/data-20/xml.po +t/data-20/test2-normalized.sgml +t/data-20/test2.pot +t/data-20/test2.sgml + +t/21-dia.t +t/data-21/extract.dia +t/data-21/extract.po-ok +t/data-21/transl.dia +t/data-21/transl.dia-ok +t/data-21/transl.po + +t/23-man.t +t/data-23/dot1 +t/data-23/dot1.fr +t/data-23/dot1.fr.po +t/data-23/dot1.pot +t/data-23/dot2 +t/data-23/dot2.err +t/data-23/dot3 +t/data-23/dot3.err +t/data-23/dot4 +t/data-23/dot4.err +t/data-23/dot5 +t/data-23/dot5.it +t/data-23/dot5.it.po +t/data-23/dot5.pot +t/data-23/escapes1 +t/data-23/escapes1.it +t/data-23/escapes1.it.po +t/data-23/escapes1.pot +t/data-23/fonts +t/data-23/fonts.en +t/data-23/fonts.en.po +t/data-23/fonts.pot +t/data-23/hyphens.1 +t/data-23/hyphens.translate.fr +t/data-23/hyphens.translate.fr.po +t/data-23/hyphens.translate.pot +t/data-23/hyphens.verbatim.fr +t/data-23/hyphens.verbatim.fr.po +t/data-23/hyphens.verbatim.pot +t/data-23/mdoc.1 +t/data-23/mdoc.fr +t/data-23/mdoc.fr.po +t/data-23/mdoc.pot +t/data-23/mixed1 +t/data-23/mixed1.fr +t/data-23/mixed2 +t/data-23/mixed2.fr +t/data-23/mixed3 +t/data-23/mixed3.fr +t/data-23/mixed4 +t/data-23/mixed4.fr +t/data-23/mixed.cfg +t/data-23/mixed.fr.po +t/data-23/mixed.pot +t/data-23/null +t/data-23/null.fr +t/data-23/null.fr.po +t/data-23/null.pot +t/data-23/quotes +t/data-23/quotes.fr +t/data-23/quotes.fr.po +t/data-23/quotes.pot +t/data-23/spaces +t/data-23/spaces.fr_latin1 +t/data-23/spaces.fr_latin1.po +t/data-23/spaces.fr_utf8 +t/data-23/spaces.fr_utf8.po +t/data-23/spaces.ja +t/data-23/spaces.ja.po +t/data-23/spaces.pot + +t/24-tex.t +t/data-24/simple-translate.out +t/data-24/simple.fr.po +t/data-24/simple-updatepo.out +t/data-24/simple.fr.tex +t/data-24/simple-gettextize.out +t/data-24/simple.pot +t/data-24/simple.tex + +t/25-xhtml.t +t/data-25/includessi.html +t/data-25/includessi_normalized.html +t/data-25/includessi.po +t/data-25/xhtml.html +t/data-25/xhtml_normalized.html +t/data-25/xhtml.po + +t/26-ini.t +t/data-26/test1.po +t/data-26/test1.ini + +t/27-xml.t +t/data-27/cdata.xml +t/data-27/cdata.po +t/data-27/general.xml +t/data-27/general.po +t/data-27/general-normalized.xml +t/data-27/comments.xml +t/data-27/comments.po +t/data-27/comments-normalized.xml +t/data-27/options.xml +t/data-27/options.po +t/data-27/options-normalized.xml + +t/28-msguntypot.t +t/data-28/test1.new.po +t/data-28/test1.new.pot +t/data-28/test1.old.pot +t/data-28/test1.po +t/data-28/test2.new.po +t/data-28/test2.new.pot +t/data-28/test2.old.pot +t/data-28/test2.po +t/data-28/test3.new.po +t/data-28/test3.new.pot +t/data-28/test3.old.pot +t/data-28/test3.po +t/data-28/test4.new.po +t/data-28/test4.new.pot +t/data-28/test4.old.pot +t/data-28/test4.po +t/data-28/test5.new.po +t/data-28/test5.new.pot +t/data-28/test5.old.pot +t/data-28/test5.po +t/data-28/test6.new.po +t/data-28/test6.new.pot +t/data-28/test6.old.pot +t/data-28/test6.po +t/data-28/test7.new.po +t/data-28/test7.new.pot +t/data-28/test7.old.pot +t/data-28/test7.po +t/data-28/test8.new.po +t/data-28/test8.new.pot +t/data-28/test8.old.pot +t/data-28/test8.po + +t/29-wml.t +t/data-29/general-normalized.wml +t/data-29/general.po +t/data-29/general.wml Index: branches/Bungo/package/bin/po4a/README.maintainers =================================================================== --- branches/Bungo/package/bin/po4a/README.maintainers (revision 0) +++ branches/Bungo/package/bin/po4a/README.maintainers (revision 2840) @@ -0,0 +1,140 @@ +This document provides some general rules to deploy a translation process +that will ease the work of maintainers (upstream and distribution +maintainers) and translators (or translation teams). + + +Translators usually fetch a POT (for a new translation) or retrieve the current +PO for their language, then they translate the untranslated strings and +update the translation of the strings marked as fuzzy. + + +Updating files +-------------- + +Translators need to know if a PO has to be updated: + * they can verify the POs in the version control system or in the + distributed archives/packages + * they can be informed by the translation teams, which automatically + check the status of the POs in various packages. +We want to avoid translators to be notified by a user reporting that +some strings are not translated even if the PO contains neither untranslated nor +fuzzy string. + + +Thus it is important to ensure that the POTs are up-to-date with the +original documents and that the POs contain the same strings as the +POTs. + + 1. Upstream maintainers should update the POTs according to the original + documents and update the POs according to these up-to-date POTs when they + distribute an archive. + + 2. If the switch to po4a was done in a distribution, the source package + should also contain up-to-date translation materials. + + 3. If the documentation is patched by the distribution, the maintainer + must not forget to update the POTs and POs. + +It is important to ensure that the translation materials are updated +automatically. + + +Architecture +------------ + +A standardized architecture of the source tree will help the translation +teams when they try to detect the POTs that need to be updated. + +Thus we recommend the following architecture: + + / + /doc/ + /doc/en/ + /doc/en/ + /doc/po4a/ + /doc/po4a/add_/ + /doc/po4a/po4a.cfg + /doc/po4a/po/ + /doc/po4a/po/.pot + /doc/po4a/po/.po + /doc/translated// + +Or, if you want to avoid a big POT and split it according to the packages, +documents, formats, or subjects, you can use the following architecture: + + / + /doc/ + /doc/en/ + /doc/en/ + /doc/po4a/ + /doc/po4a/add_/ + /doc/po4a//po4a.cfg + /doc/po4a//po/ + /doc/po4a//po/.pot + /doc/po4a//po/.po + /doc/translated// + + +It is important to avoid a build failure if a generated +translation cannot be generated (the PO is too outdated, an addendum +cannot be applied, etc.). You should therefore use wildcards or test if the file +was generated in the 'install' or 'dist' rules + + +Examples +======== + +Using po4a upstream +------------------- +When po4a is used upstream, we recommend to run po4a in the 'dist' rule. +This will update the POT and POs, and will generate the translated documents. +These translated documents can be distributed in the source archive if the +maintainer don't want to add a build dependency on po4a. You should then +add an autoconf check on po4a. It will allow you to update the documentation +if po4a is available on your system. If po4a is not available, documents +will be distributed without being synced with the original version, but the +build process won't fail. + +It is important to distribute the POT and POs in the source archive. + +A typical dist rule could then be: + +dist: + po4a .cfg + ... + +If automake is used, the following could also be used. + +dist-hook: + po4a .cfg + ... + + +Using po4a in a distribution +---------------------------- +(Debian packaging is taken as an example, you will have to adapt this to +your distribution) +To ensure that the source of a Debian package contains only up-to-date POT +and POs, you should run po4a in the 'clean' rule +of debian/rules. The translated documents can be generated in the 'build' +(or 'build-indep') rule: + +clean: + # Update the POT and POs + cd <...>/po4a && po4a --no-translations .cfg + # Delete translated documentation + rm -rf <...>/translated + +build: + # Generate the translations + cd <...>/po4a && po4a .cfg + +However, you should try to avoid distribution-specific build systems, to +ensure the portability of your software. + +Or, if using po4a-build: + +dist: + # Update the POT and POs + po4a-build --pot-only -f po4a-build.conf + $(MAKE) -C po pot Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Html.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Html.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Html.pm (revision 2840) @@ -0,0 +1,218 @@ +#!/usr/bin/perl + +# Po4a::Html.pm +# +# extract and translate translatable strings from a HTML document. +# +# This code extracts plain text between HTML tags and some "alt" or +# "title" attributes. +# +# Copyright (c) 2003 by Laurent Hausermann +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Html - convert HTML documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Html is a module to help the translation of documentation in +the HTML format into other [human] languages. + +Please note that this module is not distributed with the main po4a archive +because we don't feel it mature enough for that. If you insist on using it +anyway, check it from the CVS out. + +=cut + +package Locale::Po4a::Html; +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(new initialize); + +use Locale::Po4a::TransTractor; + +use strict; +use warnings; + +use HTML::TokeParser; + +use File::Temp; + +sub initialize {} + +sub read { + my ($self,$filename)=@_; + push @{$self->{DOCPOD}{infile}}, $filename; + $self->Locale::Po4a::TransTractor::read($filename); +} + +sub parse { + my $self=shift; + map {$self->parse_file($_)} @{$self->{DOCPOD}{infile}}; +} + +# +# Parse file and translate it +# +sub parse_file { + my ($self,$filename)=@_; + my $stream = HTML::TokeParser->new($filename) + || die "Couldn't read HTML file $filename : $!"; + + $stream->unbroken_text( [1] ); + + my @type=(); + NEXT : while (my $token = $stream->get_token) { + if($token->[0] eq 'T') { + my $text = $token->[1]; + my ($pre_spaces) = ($text =~ /^(\s*)/); + my ($post_spaces) = ($text =~ /(\s*)$/); + $text = trim($text); + if (notranslation($text) == 1) { + $self->pushline( get_tag( $token ) ); + next NEXT; + } +# FIXME : it should be useful to encode characters +# in UTF8 in the po, but converting them in HTML::Entities +# in the doc_out, translate acts both way +# so we cant do that. +# use HTML::Entities (); +# $encoded = HTML::Entities::encode($a); +# $decoded = HTML::Entities::decode($a); + #print STDERR $token->[0]; + $self->pushline( $pre_spaces . $self->translate($text, + "FIXME:0", + (scalar @type ? $type[scalar @type-1]: "NOTYPE") + ) . $post_spaces, + 'wrap' => 1 + ); + next NEXT; + } elsif ($token->[0] eq 'S') { + push @type,$token->[1]; + my $text = get_tag( $token ); + my $tag = $token->[1]; +# TODO: It would be nice to support an option to specify these +# (e.g. a list of tag.attribute) + my @trans_attr = (( $tag eq 'img' ) || ( $tag eq 'input' ) || + ( $tag eq 'area' ) || ( $tag eq 'applet')) + ? qw/title alt/ : qw/title/; + my %attr = %{$token->[2]}; + for my $a (@trans_attr) { + my $content = $attr{$a}; + if (defined $content) { + $content = trim($content); + my $translated = $self->translate( + $content, + "FIXME:0", + "${tag}_$a" + ); + $attr{$a} = $translated; + } + } + my ($closing) = ( $text =~ /(\s*\/?>)/ ); + # reconstruct the tag from scratch + delete $attr{'/'}; # Parser thinks closing / in XHTML is an attribute + $text = "<$tag"; + $text .= " $_=\"$attr{$_}\"" foreach keys %attr; + $text .= $closing; + $self->pushline( $text ); + } elsif ($token->[0] eq 'E') { + pop @type; + $self->pushline( get_tag( $token ) ); + } else { + $self->pushline( get_tag( $token ) ); + } + } +} + +sub get_tag { + my $token = shift; + my $tag = ""; + + if ($token->[0] eq 'S') { + $tag = $token->[4]; + } + if ( ($token->[0] eq 'C') || + ($token->[0] eq 'D') || + ($token->[0] eq 'T') ) { + $tag = $token->[1]; + } + if ( ($token->[0] eq 'E') || + ($token->[0] eq 'PI') ) { + $tag = $token->[2]; + } + + return $tag; +} + +sub trim { + my $s=shift; + $s =~ s/\n/ /g; # remove \n in text + $s =~ s/\r/ /g; # remove \r in text + $s =~ s/\t/ /g; # remove tabulations + $s =~ s/\s+/ /g; # remove multiple spaces + $s =~ s/^\s*//g; # remove leading spaces + $s =~ s/\s*$//g; # remove trailing spaces + return $s; +} + +# +# This method says if a string must be +# translated or not. +# To be improved with more test or regexp +# Maybe there is a way to do this in TransTractor +# for example ::^ should not be translated +sub notranslation { + my $s=shift; + return 1 if ( ($s cmp "") == 0); + return 1 if ( ($s cmp "-") == 0); + return 1 if ( ($s cmp "::") == 0); + return 1 if ( ($s cmp ":") == 0); + return 1 if ( ($s cmp ".") == 0); + return 1 if ( ($s cmp "|") == 0); + return 1 if ( ($s cmp '"') == 0); + return 1 if ( ($s cmp "'") == 0); + # don't translate entries composed of one entity + return 1 if ($s =~ /^&[^;]*;$/); + +# don't translate entries with no letters +# (happens with e.g. Hello, world ) +# ^^ +# ", " doesn't need translation + return 1 unless $s =~ /\w/; + return 0; +} + +=head1 AUTHORS + + Laurent Hausermann + +=head1 COPYRIGHT AND LICENSE + +Laurent Hausermann + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Man.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Man.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Man.pm (revision 2840) @@ -0,0 +1,2497 @@ +#!/usr/bin/perl -w + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Man - convert manual pages from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Man is a module to help the translation of documentation in +the nroff format (the language of manual pages) into other [human] +languages. + +=head1 TRANSLATING WITH PO4A::MAN + +This module tries pretty hard to make translator's life easier. For that, +the text presented to translators isn't a verbatim copy of the text found +in the man page. Indeed, the cruder parts of the nroff format are hidden, so +that translators can't mess up with them. + +=head2 Text wrapping + +Unindented paragraphs are automatically rewrapped for the translator. This +can lead to some minor difference in the generated output, since the +rewrapping rules used by groff aren't very clear. For example, two spaces +after a parenthesis are sometimes preserved. + +Anyway, the difference will only be about the position of the extra spaces +in wrapped paragraph, and I think it's worth. + +=head2 Font specification + +The first change is about font change specifications. In nroff, there are +several ways to specify if a given word should be written in small, bold or +italics. In the text to translate, there is only one way, borrowed from the +POD (Perl online documentation) format: + +=over + +=item IEtextE -- italic text + +equivalent to \fItext\fP or ".I text" + +=item BEtextE -- bold text + +equivalent to \fBtext\fP or ".B text" + +=item REtextE -- roman text + +equivalent to \fRtext\fP + +=item CWEtextE -- constant width text + +equivalent to \f(CWtext\fP or ".CW text" + +=back + +Remark: The CW face is not available for all groff devices. It is not +recommended to use it. It is provided for your convenience. + +=head2 Automatic characters transliteration + +Po4a automatically transliterate some characters to ease the translation +or the review of the translation. +Here is the list of the transliterations: + +=over + +=item hyphens + +Hyphens (-) and minus signs (\-) in man pages are all transliterated +as simple dashes (-) in the PO file. Then all dash are transliterated into +roff minus signs (\-) when the translation is inserted into the output +document. + +Translators can force an hyphen by using the roff glyph '\[hy]' in their +translations. + +=item non-breaking spaces + +Translators can use non-breaking spaces in their translations. These +non-breaking spaces (0xA0 in latin1) will be transliterated into a roff +non-breaking space ('\ '). + +=item quotes transliterations + +`` and '' are respectively tranliterated into \*(lq and \*(rq. + +To avoid these transliterations, translators can insert a zero width roff +character (i.e., using `\&` or '\&' respectively). + +=back + +=head2 Putting 'E' and 'E' in translations + +Since these chars are used to delimit parts under font modification, you +can't use them verbatim. Use EEltE and EEgtE instead (as in +POD, one more time). + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +These are this module's particular options: + +=over + +=item B + +Activate debugging for some internal mechanisms of this module. +Use the source to see which parts can be debugged. + +=item B + +Increase verbosity. + +=item B + +This option permits to change the behavior of the module when it encounter +a .de, .ie or .if section. It can take the following values: + +=over + +=item I + +This is the default value. +The module will fail when a .de, .ie or .if section is encountered. + +=item I + +Indicates that the .de, .ie or .if sections must be copied as is +from the original to the translated document. + +=item I + +Indicates that the .de, .ie or .if sections will be proposed for the +translation. +You should only use this option if a translatable string is +contained in one of these section. Otherwise, I +should be preferred. + +=back + +=item B + +This option specifies that the file was generated, and that po4a should not +try to detect if the man pages was generated from another format. +This permits to use po4a on generated man pages. +This option does not take any argument. + +=item B + +This option is only useful for mdoc pages. + +It selects a stricter support of the mdoc format by telling po4a not to +translate the 'NAME' section. +mdoc pages whose 'NAME' section is translated won't generate any header or +footer. + +According to the groff_mdoc page, the NAME, SYNOPSIS and DESCRIPTION +sections are mandatory. +There are no known issues with translated SYNOPSIS or DESCRIPTION section, +but you can also specify these sections this way: + -o mdoc=NAME,SYNOPSIS,DESCRIPTION + +This mdoc issue can also be solved with an addendum like this one: + PO4A-HEADER:mode=before;position=^.Dd + .TH DOCUMENT_TITLE 1 "Month day, year" OS "Section Name" + + +=back + +The following options permit to specify the behavior of a new macro +(defined with a .de request), or of a macro not supported by po4a. +They take as argument a comma-separated list of macros. +For example: + + -o noarg=FO,OB,AR -o translate_joined=BA,ZQ,UX + +Note: if a macro is not supported by po4a and if you consider that it is a +standard roff macro, you should submit it to the po4a development team. + +=over + +=item B + +B indicates that this macro (at its arguments) don't have to +be translated. + +=item B + +B is like B, except that po4a will verify that no +argument is added to this macro. + +=item B + +B indicates that po4a must propose to translate the +arguments of the macro. + +=item B + +With B, the arguments will also be proposed for the +translation, except that each one will be translated separately. + +=item B + +This option takes as argument a list of comma-separated couples +I:I, where I and I are commands that delimit +the begin and end of a section that should not be rewrapped. + +Note: no test is done to ensure that an I command matches its +I command; any ending command stop the no_wrap mode. +If you have a I (respectively I) macro that has no I +(respectively I), you can specify an existing I (like fi) or +I (like nf) as a counterpart. +These macros (and their arguments) wont be translated. + +=item B + +This option specifies a list of comma-separated macros that must +not split the current paragraph. The string to translate will then contain +I.bar baz quxE quux>, where I is the command that +should be inlined, and I its arguments. + +=item B + +This option indicates how po4a should behave when an unknown macro is found. +By default, po4a fails with a warning. +It can take the following values: B (the default value), +B, B, B, or B (see above +for an explanation of these values). + +=back + +=head1 AUTHORING MAN PAGES COMPLIANT WITH PO4A::MAN + +This module is still very limited, and will always be, because it's not a +real nroff interpreter. It would be possible to do a real nroff +interpreter, to allow authors to use all the existing macros, or even to +define new ones in their pages, but we didn't want to. It would be too +difficult, and we thought it wasn't necessary. We do think that if +manpages' authors want to see their productions translated, they may have to +adapt to ease the work of translators. + +So, the man parser implemented in po4a have some known limitations we are +not really inclined to correct, and which will constitute some pitfalls +you'll have to avoid if you want to see translators taking care of your +documentation. + +=head2 Don't program in nroff + +nroff is a complete programming language, with macro definition, +conditionals and so on. Since this parser isn't a fully featured nroff +interpreter, it will fail on pages using these facilities (There are about +200 such pages on my box). + +=head2 Use the plain macro set + +There are still some macros which are not supported by po4a::man. This is +only because I failed to find any documentation about them. Here is the +list of unsupported macros used on my box. Note that this list isn't +exhaustive since the program fails on the first encountered unsupported +macro. If you have any information about some of these macros, I'll +happily add support for them. Because of these macros, about 250 pages on +my box are inaccessible to po4a::man. + + .. ." .AT .b .bank + .BE ..br .Bu .BUGS .BY + .ce .dbmmanage .do .En + .EP .EX .Fi .hw .i + .Id .l .LO .mf + .N .na .NF .nh .nl + .Nm .ns .NXR .OPTIONS .PB + .pp .PR .PRE .PU .REq + .RH .rn .S< .sh .SI + .splitfont .Sx .T .TF .The + .TT .UC .ul .Vb .zZ + +=head2 Hiding text from po4a + +Sometimes, the author knows that some parts are not translatable, and +should not be extracted by po4a. For example, an option may accept an +I argument, and I may also appear as the last item of a +list. In the first case, I should be not be translatable. And in +the second case, I should be translated. + +In such case, the author can avoid po4a to extract some strings, using +some special groff constructs: + + .if !'po4a'hide' .B other + +(this will require the B<-o groff_code=verbatim> option) + +A new macro can also be defined to automate this: + .de IR_untranslated + . IR \\$@ + .. + + .IR_untranslated \-q ", " \-\-quiet + +(this will require the options B<-o groff_code=verbatim> and +B<-o untranslated=IR_untranslated>; with this construct, the B<.if +!'po4a'hide'> conditional is not strictly needed since po4a will not parse +the internal of the macro definition) + +or using an alias: + .als IR_untranslated IR + + .IR_untranslated \-q ", " \-\-quiet + +(this will require the B<-o untranslated=als,IR_untranslated> option) + +=head2 Conclusion + +To summarise this section, keep simple, and don't try to be clever while +authoring your man pages. A lot of things are possible in nroff, and not +supported by this parser. For example, don't try to mess with \c to +interrupt the text processing (like 40 pages on my box do). Or, be sure to +put the macro arguments on the same line that the macro itself. I know that +it's valid in nroff, but would complicate too much the parser to be +handled. + +Of course, another possibility is to use another format, more translator +friendly (like POD using po4a::pod, or one of the XML familly like SGML), +but thanks to po4a::man it isn't needed anymore. That being said, if the +source format of your documentation is POD, or XML, it may be clever to +translate the source format and not this generated one. In most cases, +po4a::man will detect generated pages and issue a warning. It will even +refuse to process POD generated pages, because those pages are perfectly +handled by po4a::pod, and because their nroff counterpart defines a lot of +new macros I didn't want to write support for. On my box, 1432 of the 4323 +pages are generated from POD and will be ignored by po4a::man. + +In most cases, po4a::man will detect the problem and refuse to process the +page, issuing an adapted message. In some rare cases, the program will +complete without warning, but the output will be wrong. Such cases are +called "bugs" ;) If you encounter such case, be sure to report this, along +with a fix when possible... + +=head1 STATUS OF THIS MODULE + +This module can be used for most of the existing man pages. + +Some tests are regularly run on Linux boxes: + +=over 4 + +=item * + +one third of the pages are refused because they were generated from +another format supported by po4a (e.g. POD or SGML). + +=item * + +10% of the remaining pages are rejected with an error (e.g. a +groff macro is not supported). + +=item * + +Then, less than 1% of the pages are accepted silently by po4a, but with +significant issues (i.e. missing words, or new words inserted) + +=item * + +The other pages are usually handled without differences more important +than spacing differences or line rewrapped (font issues in less than 10% of +the processed pages). + +=back + +=head1 SEE ALSO + +L, +L, +L + +=head1 AUTHORS + + Denis Barbier + Nicolas François + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002-2008 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +package Locale::Po4a::Man; +use DynaLoader; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor DynaLoader); +@EXPORT = qw();# new initialize); + +# Try to use a C extension if present. +eval('bootstrap Locale::Po4a::Man "0.30"'); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +use File::Spec; +use Getopt::Std; + +my %macro; # hash of known macro, with parsing sub. See end of this file +my %default_macro; # The default known macros, when no options are used. + +# A font start by \f and is followed either by +# [.*] - a font name within brackets (e.g. [P], [A_USER_FONT]) +# (.. - a parenthesis followed by two char (e.g. "(CW") +# . - a single char (e.g. B, I, R, P, 1, 2, 3, 4, etc.) +my $FONT_RE = "\\\\f(?:\\[[^\\]]*\\]|\\(..|[^\\(\\[])"; + +# Variable used to identify non breaking spaces. +# These non breaking spaces are used to ease the parsing, and a +# translator can use them in her translation (and they will be translated +# into the groff non-breaking space. +my $nbs; + +# Indicate if the page uses the mdoc macros +my $mdoc_mode = 0; + +my $unknown_macros = undef; + +######################### +#### DEBUGGING STUFF #### +######################### +my %debug; +# The following debug options can be set with '-o debug=...': +# * splitargs see how macro args are separated +# * pretrans see pre-conditioning of translation +# * postrans see post-conditioning of translation +# * fonts see font modifier handling + + +######## CONFIG ######### +# This variable indicates the behavior of the module when a .de, .if or +# .ie is encountered. +my $groff_code; +# %no_wrap_begin and %no_wrap_end are lists of macros that respectively +# begins and ends a no_wrap paragraph. +# Any ending macro will end the no_wrap paragraph started by any beginning +# macro. +my %no_wrap_begin; +my %no_wrap_end; +# List of macros that should be inlined (with E<.xx ...>) +my %inline; +# The default list of inlined macros (when no options are used) +my %default_inline; +# This variable indicates whether po4a should try to detect the generated +# files. +my $allow_generated; +# This hash indicates section name that should not be translated in mdoc +# mode. +# The groff's mdoc processor requires the NAME section, otherwise headers +# and footers of the pages are not generated. +# The mdoc_groff man page indicates that NAME, SYNOPSIS and DESCRIPTION +# are mandatory. +my %mdoc; +sub initialize { + my $self = shift; + my %options = @_; + + $self->{options}{'debug'}=''; + $self->{options}{'verbose'}=''; + $self->{options}{'groff_code'}=''; + $self->{options}{'untranslated'}=''; + $self->{options}{'noarg'}=''; + $self->{options}{'translate_joined'}=''; + $self->{options}{'translate_each'}=''; + $self->{options}{'no_wrap'}=''; + $self->{options}{'inline'}=''; + $self->{options}{'generated'}=''; + $self->{options}{'mdoc'}=''; + $self->{options}{'unknown_macros'}=''; + + foreach my $opt (keys %options) { + if (defined $options{$opt}) { + die wrap_mod("po4a::man", + dgettext("po4a", "Unknown option: %s"), $opt) + unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options{$opt}; + } + } + + %debug = (); + if (defined $options{'debug'}) { + foreach ($options{'debug'}) { + $debug{$_} = 1; + } + } + + $groff_code = "fail"; + if (defined $options{'groff_code'}) { + unless ($options{'groff_code'} =~ m/fail|verbatim|translate/) { + die wrap_mod("po4a::man", dgettext("po4a", + "Invalid 'groff_code' value. Must be one of 'fail', 'verbatim', 'translate'.")); + } + $groff_code = $options{'groff_code'}; + } + + if (%default_macro) { + %macro = %default_macro; + } else { + %default_macro = %macro + } + if (defined $options{'untranslated'}) { + foreach (split(/,/, $options{'untranslated'})) { + $macro{$_} = \&untranslated; + } + } + if (defined $options{'noarg'}) { + foreach (split(/,/, $options{'noarg'})) { + $macro{$_} = \&noarg; + } + } + if (defined $options{'translate_joined'}) { + foreach (split(/,/, $options{'translate_joined'})) { + $macro{$_} = \&translate_joined; + } + } + if (defined $options{'translate_each'}) { + foreach (split(/,/, $options{'translate_each'})) { + $macro{$_} = \&translate_each; + } + } + + %no_wrap_begin = ( + 'nf' => 1, + 'EX' => 1, + 'EQ' => 1 + ); + %no_wrap_end = ( + 'fi' => 1, + 'EE' => 1, + 'EN' => 1 + ); + if (defined $options{'no_wrap'}) { + foreach (split(/,/, $options{'no_wrap'})) { + if ($_ =~ m/^(.*):(.*)$/) { + $no_wrap_begin{$1} = 1; + $no_wrap_end{$2} = 1; + } else { + die wrap_mod("po4a::man", dgettext("po4a","The no_wrap parameters must be a set of comma-separated begin:end couples.\n")); + } + } + } + + if (%default_inline) { + %inline = %default_inline; + } else { + %default_inline = %inline + } + if (defined $options{'inline'}) { + foreach (split(/,/, $options{'inline'})) { + $inline{$_} = 1; + } + } + + $allow_generated = 0; + if (defined $options{'generated'}) { + $allow_generated = 1; + } + + %mdoc = (); + if (defined $options{'mdoc'}) { + if ($options{'mdoc'} eq 1) { + $mdoc{"NAME"} = 1; + } else { + foreach (split(/,/, $options{'mdoc'})) { + $mdoc{$_} = 1; + } + } + } + + $unknown_macros = undef; + if (defined $options{'unknown_macros'}) { + if ($options{'unknown_macros'} eq "failed") { + $unknown_macros = undef; + } elsif ($options{'unknown_macros'} eq "untranslated") { + $unknown_macros = \&untranslated; + } elsif ($options{'unknown_macros'} eq "noarg") { + $unknown_macros = \&noarg; + } elsif ($options{'unknown_macros'} eq "translate_joined") { + $unknown_macros = \&translate_joined; + } elsif ($options{'unknown_macros'} eq "translate_each") { + $unknown_macros = \&translate_each; + } else { + die wrap_mod("po4a::man", dgettext("po4a", + "Invalid 'unknown_macros' value. Must be one of:\n"). + "failed untranslated noarg translate_joined translate_each\n"); + } + } +} + +my @comments = (); +my @next_comments = (); +# This function returns the next line of the document being parsed +# (and its reference). +# It overload the Transtractor shiftline to handle: +# - font requests (.B, .I, .BR, .BI, ...) +# because these requests can be present in a paragraph (handled +# in the parse subroutine), or in argument (on the next line) +# of some other request (for example .TP) +# - font size requests (.SM,.SB) (not done yet) +# - input escape (\ at the end of a line) +sub shiftline { + my $self = shift; + # call Transtractor's shiftline +NEW_LINE: + my ($line,$ref) = $self->SUPER::shiftline(); + + if (!defined $line) { + # end of file + return ($line,$ref); + } + + # Do as few treatments as possible with the .de, .ie and .if sections + if ($line =~ /^\.\s*(if|ie|de)/) { + chomp $line; + return ($line,$ref); + } + + # Handle some escapes + # * reduce the number of \ in macros + if ($line =~ /^\\?[.']/) { + # The first backslash is consumed while the macro is read. + $line =~ s/\\\\/\\/g; + } + # * \\ is equivalent to \e, which is less error prone for the rest + # of the module (e.g. when searching for a font : \f, whe don't + # want to match \\f) + $line =~ s/\\\\/\\e/g; + # * \. is just a dot (this can even be use to introduce a macro) + $line =~ s/\\\././g; + + chomp $line; + if ($line =~ m/^(.*?)(?:(?SUPER::shiftline(); + chomp($l2); + if ($line =~ /^(\.[BI])\s*$/) { + if ($l2 =~ /^[.'][\t ]*([BI]|BI|BR|IB|IR|RB|RI)(?:[\t ]|\s*$)/) { + my $font = $line; + $font =~ s/^\.([BI])\s*$/$1/; + $insert_font = "\\f$font$insert_font"; + $line = $l2; + $ref = $r2; + } elsif ($l2 =~ /^[.'][\t ]*(SH|TP|TQ|P|PP|LP)(?:[\t ]|\s*$)/) { + $line =~ s/^\.([BI])\s*$/$insert_font\\f$1/; + $self->SUPER::unshiftline($l2,$r2); + } elsif ($l2 =~ /^([.'][\t ]*(?:IP)[\t ]+"?)(.*)$/) { + # Install the font modifier into the next line + # after a possible quote (") + my $macro = $1; + my $arg = $2; + $line =~ /^\.([BI])\s*$/; + $line = $macro."$insert_font\\f$1".$arg; + $ref = $r2; + } elsif ($l2 =~ /^[.']/) { + warn wrap_ref_mod($ref, "po4a::man", dgettext("po4a", + "Font modifiers followed by a command may disturb ". + "po4a. You should either remove the font modifier ". + "'%s', or integrate a \\f font modifier in the ". + "following command ('%s'), but continuing anyway." + ), $line, $l2); + $line = "PO4A-INLINE:$line:PO4A-INLINE"; + $self->SUPER::unshiftline($l2,$r2); + } else { + # convert " to the groff's double quote glyph; it will be + # converted back to " in pre_trans. It is needed because + # otherwise, these quotes will be taken as arguments + # delimiters. + $l2 =~ s/"/\\(dq/g; + # append this line to the macro, with surrounding quotes, so + # that the line appear as an uniq argument. + $line .= ' "'.$l2.'"'; + } + } else { + $line =~ s/\\$//; + $line .= $l2; + } + } + # Detect non-wrapped paragraphs + # This must be done before handling the .B, .RI ... font requests + $line =~ s/^($FONT_RE)(\s+)/$2$1/; + + $line .= "\n"; + + # Handle font requests here + if ($line =~ /^[.'][\t ]*([BI]|BI|BR|IB|IR|RB|RI)(?:(?: +|\t)(.*)|)$/) { + my $macro = $1; + my $arguments = $2; + my @args = splitargs($ref,$arguments); + if ($macro eq 'B' || $macro eq 'I') { + # To keep the space(s), we must introduce some \& + @args = map { $_ =~ s/^(\s*)$/\\&$1\\&/s; $_ } @args; + my $arg=join(" ",@args); + $arg =~ s/^ +//; + this_macro_needs_args($macro,$ref,$arg); + $line = "$insert_font\\f$macro".$arg."\\fR\n"; + $insert_font = ""; + } + # .BI bold alternating with italic + # .BR bold/roman + # .IB italic/bold + # .IR italic/roman + # .RB roman/bold + # .RI roman/italic + if ($macro eq 'BI' || $macro eq 'BR' || $macro eq 'IB' || + $macro eq 'IR' || $macro eq 'RB' || $macro eq 'RI' ) { + # num of seen args, first letter of macro name, second one + my ($i,$a,$b)=(0,substr($macro,0,1),substr($macro,1)); + $line = join("", map { $i++ % 2 ? + "\\f$b$_" : + "\\f$a$_" + } @args)."\\fR\n"; + if ($i eq 0) { + # If a .BI is used without argument, we must insert a + # \fI\fR. The \fR was inserted previously. + $line = "\\f$b$line"; + } + } + + if (length $insert_font) { + $line =~ s/\n$//; + $line = "$insert_font$line\\fR\n"; + } + + if ($line =~ /^(.*)\\c(\\f.)?\s*\\fR\n/) { + my $begin = $1; + + my ($l2,$r2)=$self->SUPER::shiftline(); + if ($l2 =~ /^[.']/) { + $self->SUPER::unshiftline($l2,$r2); + } else { + $l2 =~ s/\s*$//s; + $line = "$begin\\fR$l2\n"; + } + } + } + + return ($line,$ref); +} + +# Overload Transtractor's pushline. +# This pushline first push comments (if there are comments for the +# current line, and the line is not empty), and then push the line. +sub pushline { + my ($self, $line) = (shift, shift); + if ($line !~ m/^\s*$/) { + # add comments + foreach my $c (@comments) { + # comments are pushed (maybe at the wrong place). + $self->SUPER::pushline($self->r(".\\\"$c\n")); + } + @comments = (); + } + + $self->SUPER::pushline($line); +} + +# The default unshiftline from Transtractor may fail because shiftline +# is overloaded +sub unshiftline { + die wrap_mod("po4a::man", dgettext("po4a", + "The unshiftline is not supported for the man module. ". + "Please send a bug report with the groff page that generated ". + "this error.")); +} + +############################################### +#### FUNCTION TO TRANSLATE OR NOT THE TEXT #### +############################################### +sub pushmacro { + my $self=shift; + if (scalar @_) { + # Do quote the arguments containing spaces, as it should. + + # but do not do so if they already contain quotes and escaped spaces + # For example, cdrdao(1) uses: + # .IP CATALOG\ "ddddddddddddd" (Here, the quote have to be displayed) + # Adding extra quotes as in: + # .IP "CATALOG\ "ddddddddddddd"" + # results in two args: 'CATALOG\ ' and 'ddddddddddddd""' + $self->pushline(join(" ",map { + # Replace double quotes by \(dq (double quotes could be + # taken as an argument delimiter). + # Only quotes not preceded by \ are taken into account + # (\" introduces a comment). + s/(?pushline("\n"); + } +} +sub this_macro_needs_args { + my ($macroname,$ref,$args)=@_; + unless (length($args)) { + die wrap_ref_mod($ref, "po4a::man", dgettext("po4a", + "macro %s called without arguments. ". + "Even if placing the macro arguments on the next line is authorized ". + "by man(7), handling this would make the po4a parser too complicate. ". + "Please simply put the macro args on the same line." + ), $macroname); + } +} + +sub pre_trans { + my ($self,$str,$ref,$type)=@_; + # Preformatting, so that translators don't see + # strange chars + my $origstr=$str; + print STDERR "pre_trans($str)=" + if ($debug{'pretrans'}); + + # Do as few treatments as possible with the .de, .ie and .if sections + if (defined $self->{type} && $self->{type} =~ m/^(ie|if|de)$/) { + return $str; + } + + # Note: if you want to implement \c support, the gdb man page is your playground + if ( not defined $self->{type}) { + $str =~ s/(\G|^(?:.*?)\n|^) # Last position, or begin of a line + ([ \t]*[^.'][^\n]*(?/E/sg; + $str =~ s//sg; + $str =~ s/EEgt>/E/g; # could be done in a smarter way? + + while ($str =~ m/^(.*)PO4A-INLINE:(.*?):PO4A-INLINE(.*)$/s) { + my ($t1,$t2, $t3) = ($1, $2, $3); + $str = "$1E<$2>"; + if ($mdoc_mode) { + # When a punctuation sign must be joined to an argument, mdoc + # permits to use such a construct: + # .Ar file1 , file2 , file3 ) . + # Here, we move the punctuation out of the E<...> tag. + # This is reverted in post_trans. + # FIXME: To be checked with the French punctuation + while ($str =~ m/(?/s) { + $str =~ s/(?/>$1/s; + } + } + if (defined $t3 and length $t3) { + $t3 =~ s/^\n//s; + $str .= "\n$t3"; + } + } + + # simplify the fonts for the translators + if (defined $self->{type} && $self->{type} =~ m/^(SH|SS)$/) { + set_regular("B"); + } + $str = do_fonts($str, $ref); + if (defined $self->{type} && $self->{type} =~ m/^(SH|SS)$/) { + set_regular("R"); + } + + # After the simplification, the first char can be a \n. + # Simply push these newlines before the translation, but make sure the + # resulting string is not empty (or an additional line will be + # added). + if ($str =~ /^(\n+)(.+)$/s) { + $self->pushline($1); + $str = $2; + } + + unless ($mdoc_mode) { + # Kill minus sign/hyphen difference. + # Aestetic of printed man pages may suffer, but: + # * they are translator-unfriendly + # * they break when using utf8 (for obscure reasons) + # * they forbid the searches, since keybords don't have hyphen key + # * they forbid copy/paste, since options need minus sign, not hyphen + $str =~ s|\\-|-|sg; + + # Groff bestiary + $str =~ s/\\\*\(lq/``/sg; + $str =~ s/\\\*\(rq/''/sg; + $str =~ s/\\\(dq/"/sg; + } + + # non-breaking spaces + # some non-breaking spaces may have been added during the parsing + $str =~ s/\Q$nbs/\\ /sg; + + print STDERR "$str\n" if ($debug{'pretrans'}); + return $str; +} + +sub post_trans { + my ($self,$str,$ref,$type,$wrap)=@_; + my $transstr=$str; + + print STDERR "post_trans($str)=" + if ($debug{'postrans'}); + + # Do as few treatments as possible with the .de, .ie and .if sections + if (defined $self->{type} && $self->{type} =~ m/^(ie|if|de)$/) { + return $str; + } + + unless ($mdoc_mode) { + # Post formatting, so that groff see the strange chars + $str =~ s|\\-|-|sg; # in case the translator added some of them manually + # change hyphens to minus signs + # (this shouldn't be done for \s- font size modifiers) + # nor on .so/.mso args + unless (defined $self->{type} && $self->{type} =~ m/^m?so$/) { + my $tmp = ""; + while ($str =~ m/^(.*?)-(.*)$/s) { + my $begin = $1; + $str = $2; + my $tmp2 = $tmp.$begin; + if ( ($begin =~ m/(?)?$/s) + or ($tmp2 =~ m/(?]*)\n([^>]*>)/$1 $2/gs; + + # No . or ' on first char, or nroff will think it's a macro + # * at the beginning of a paragraph, add \& (zero width space) at + # the beginning of the line + if (not defined $self->{type}) { + # Only do it on regular text, because + # his doesn't work after a TS (this macros shift + # lines, which may contain macros) + # or for the .ta arguments (e.g. .ta .5i 3i) + $str =~ s/^((?: + (?:CW|[RBI])< + |$FONT_RE + )? + [.'] + )/\\&$1/mgx; + } elsif ($self->{type} =~ m/^(TP|TQ)$/) { + # But it is also needed for some type (e.g. TP, if followed by a + # font macro) + # This regular expression is the same as above + $str =~ s/^((?:(?:CW|[RBI])<|$FONT_RE)?[.'])/\\&$1/mg; + } + # * degraded mode, doesn't work for the first line of a paragraph + $str =~ s/\n([.'])/ $1/mg; + + # Change ascii non-breaking space to groff one + my $nbs_out = get_out_nbs($self->get_out_charset); + $str =~ s/\Q$nbs_out/\\ /sg if defined $nbs_out; + # No nbsp (said "\ " in groff on the last pos of the line, or groff adds + # an extra space + $str =~ s/\\ \n(?=.)/\\ /sg; + + # Make sure we compute internal sequences right. + # think about: B EZA E> + while ($str =~ m/^(.*)(CW|[RBI])<(.*)$/s) { + my ($done,$rest)=($1."\\f$2",$3); + $done =~ s/CW$/\(CW/; + my $lvl=1; + while (length $rest && $lvl > 0) { + my $first=substr($rest,0,1); + if ($first eq '<') { + $lvl++; + } elsif ($first eq '>') { + $lvl--; + } + $done .= $first if ($lvl > 0); + $rest=substr($rest,1); + } + die wrap_ref_mod($ref||$self->{ref}, "po4a::man", dgettext("po4a","Unbalanced '<' and '>' in font modifier. Faulty message: %s"),$str) + if ($lvl > 0); + # Return to the regular font + $done .= "\\fP$rest"; + $str=$done; + } + + while ($str =~ m/^(.*?)E<([.'][\t ]*.*?(?(.*)$/s) { + my ($t1, $t2, $t3) = ($1,$2,$3); + $t1 =~ s/ +$//s; + $t2 =~ s/\n/ /gs; + if ($mdoc_mode) { + # restore the punctuation inside the line (see pre_trans) + if ($t3 =~ s/^([.,;:\)\]]+)//s) { + my $punctuation = $1; + $punctuation =~ s/([.,;:\)\]])/$1 /; + $t2 .= " $punctuation"; + } + } + $t3 =~ s/^ +//s; + if ($wrap) { + # The no-wrap case should be checked + $t1 =~ s/\n$//s; + } + $str = $t1; + if (length $t1) { + $t1 =~ s/\n$//s; + $str = "$t1\n"; + } + $str .= $t2; + if (defined $t3 and length $t3) { + $t3 =~ s/^\n//s; + $str.= "\n$t3"; + } + } + my $str2 = $str; + $str2 =~ s/E<[gl]t>//g; + die wrap_ref_mod($ref||$self->{ref}, "po4a::man", + dgettext("po4a","Unknown '<' or '>' sequence. ". + "Faulty message: %s"),$str) + if $str2 =~ /[<>]/; + $str =~ s/E/>/mg; + $str =~ s/E/{type}) { + $str =~ s/(?{ref}); + return $str; + } + + # If a string is quoted, only translate the argument between the + # quotes. + if ($options{'wrap'} or $str !~ m/\n/s) { + if ($str =~ m/^\"(.*)\"$/s and $1 !~ m/(?translate($1, $ref, $type, %options).'"'; + $str =~ s/\n"$/"\n/s; + return $str; + } + } + + $str=pre_trans($self,$str,$ref||$self->{ref},$type); + $options{'comment'} .= join('\n', @comments); + # Translate this + $str = $self->SUPER::translate($str, + $ref||$self->{ref}, + $type || $self->{type}, + %options); + if ($options{'wrap'}) { + my (@paragraph); + @paragraph=split (/\n/,$str); + if (defined ($paragraph[0]) && $paragraph[0] eq '') { + shift @paragraph; + } + $str = join("\n",@paragraph)."\n"; + } + $str=post_trans($self,$str,$ref||$self->{ref},$type, $options{'wrap'}); + return $str; +} + +# shortcut +sub t { + return $_[0]->translate($_[1]); +} + +# shortcut. +# As a rule of thumb, I do not recode macro names, unless they may be +# followed by other characters. +sub r { + my $self = shift; + my $str = shift; + + # non-breaking spaces + # some non-breaking spaces may have been added during the parsing + $str =~ s/\Q$nbs/\\ /sg; + + return $self->recode_skipped_text($str); +} + + +sub do_paragraph { + my ($self,$paragraph,$wrapped_mode) = (shift,shift,shift); + + # Following needed because of 'ft' (at least, see ft macro below) + unless ($paragraph =~ m/\n$/s) { + my @paragraph = split(/\n/,$paragraph); + + $paragraph .= "\n" + unless scalar (@paragraph) == 1; + } + + $self->pushline( $self->translate($paragraph,$self->{ref},"Plain text", + "wrap" => ($wrapped_mode eq 'YES') ) ); +} + +############################# +#### MAIN PARSE FUNCTION #### +############################# +sub parse{ + my $self = shift; + my ($line,$ref); + my ($paragraph)=""; # Buffer where we put the paragraph while building + my $wrapped_mode='YES'; # Should we wrap the paragraph? Three possible values: + # YES: do wrap + # NO: don't wrap because this paragraph contains indented lines + # this status disapear after the end of the paragraph + # MACRONO: don't wrap because we saw the nf macro. It stays so + # until the next fi macro. + + + # We want to change the non-breaking space according to the input + # document charset + $nbs = get_in_nbs($self->{TT}{'file_in_charset'}); + + LINE: + undef $self->{type}; + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { +# print STDERR "line=$line;ref=$ref"; + chomp($line); + $self->{ref}="$ref"; +# print STDERR "LINE=$line<<\n"; + + + if ($line =~ /^[.']/) { + die wrap_mod("po4a::man", dgettext("po4a", "Unparsable line: %s"), $line) + unless ($line =~ /^([.']+\\*?)(\\["#])(.*)/ || + $line =~ /^([.'])(\S*)(.*)/); + my $arg1=$1; + $arg1 .= $2; + my $macro=$2; + my $arguments=$3; + + if ($inline{$macro}) { + $paragraph .= "PO4A-INLINE:".$line.":PO4A-INLINE\n"; + goto LINE; + } + + # Split on spaces for arguments, but not spaces within double quotes + my @args=(); + push @args,$arg1; + if ($macro =~ /^(?:ta|TP|ie|if|de)$/) { + # The number of spaces may be critical for the 'ta' macro, + # and there is no need to split the arguments. + push @args, $arguments; + } else { + push @args, splitargs($ref,$arguments); + } + + + if (length($paragraph)) { + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = $wrapped_mode eq 'NO' ? 'YES' : $wrapped_mode; + } + + # Special case: Don't change these lines + # .\" => comments + # .\# => comments + # ." => comments + # . => empty point on the line + # .tr abcd... + # => substitution like Perl's tr/ac/bd/ on output. + if ($macro eq '\\"' || $macro eq '' || $macro eq 'tr' || + $macro eq '"' || $macro eq '\\#') { + $self->pushline($self->r($line)."\n"); + goto LINE; + } + # Special case: + # .nf => stop wrapped mode + # .fi => wrap again + if ($no_wrap_begin{$macro} or $no_wrap_end{$macro}) { + if ($no_wrap_end{$macro}) { + $wrapped_mode='YES'; + } else { + $wrapped_mode='MACRONO'; + } + $self->pushline($self->r($line)."\n"); + goto LINE; + } + + # SH resets the wrapping (in addition to starting a section) + if ($macro eq 'SH') { + $wrapped_mode='YES'; + } + + unshift @args,$self; + # Apply macro + $self->{type}=$macro; + + if (defined ($macro{$macro})) { + &{$macro{$macro}}(@args); + } else { + if (defined $unknown_macros) { + &{$unknown_macros}(@args); + } else { + $self->pushline($self->r($line)."\n"); + die wrap_ref_mod($ref, "po4a::man", dgettext("po4a", + "Unknown macro '%s'. Remove it from the document, or refer to the Locale::Po4a::Man manpage to see how po4a can handle new macros."), $line); + } + } + + } elsif ($line =~ /^ +[^. ]/) { + # (Lines containing only spaces are handled as empty lines) + # Not a macro, but not a wrapped paragraph either + $wrapped_mode = $wrapped_mode eq 'YES' ? 'NO' : $wrapped_mode; + $paragraph .= $line."\n"; + } elsif ($line =~ /^[^.].*/ && $line !~ /^ *$/) { + # (Lines containing only spaces are handled latter as empty lines) + if ($line =~ /^\\"/) { + # special case: the line is entirely a comment, keep the + # comment. + # NOTE: comment could also be found in the middle of a line. + # From info groff: + # Escape: \": Start a comment. Everything to the end of the + # input line is ignored. + $self->pushline($self->r($line)."\n"); + goto LINE; + } elsif ($line =~ /^\\#/) { + # Special groff comment. Do not keep the new line + goto LINE; + } else { + # Not a macro + # * first, try to handle some "output line continuation" (\c) + $paragraph =~ s/\\c *(($FONT_RE)?)\n?$/$1/s; + # * append the line to the current paragraph + $paragraph .= $line."\n"; + } + } else { #empty line, or line containing only spaces + if (length($paragraph)) { + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + } + $wrapped_mode = $wrapped_mode eq 'NO' ? 'YES' : $wrapped_mode; + $self->pushline($line."\n"); + } + + # finally, we did not reach the end of the paragraph. The comments + # belong to the current paragraph. + push @comments, @next_comments; + @next_comments = (); + + # Reinit the loop + ($line,$ref)=$self->shiftline(); + undef $self->{type}; + } + + if (length($paragraph)) { + do_paragraph($self,$paragraph,$wrapped_mode); + $wrapped_mode = $wrapped_mode eq 'NO' ? 'YES' : $wrapped_mode; + $paragraph=""; + } + + # flush the last comments + push @comments, @next_comments; + @next_comments = @comments; + @comments = (); + for my $c (@next_comments) { + $self->pushline($self->r(".\\\"$c\n")); + } + + # reinitialize the module + @next_comments = (); + set_regular("R"); + set_font("R"); + set_font("R"); + $mdoc_mode = 0; +} # end of main + +# Cache the results of get_in_nbs and get_out_nbs +{ + my $last_in_charset; + my $last_in_nbs; + +# get_in_nbs(charset) +# Return the representation of a non breaking space in the input charset +# (given in argument). +# or PO4A:VERY_IMPROBABLE_STRING_USEDFOR_NON-BREAKING-SPACES if this +# character doesn't exist in this charset. + sub get_in_nbs() { + my $charset = shift; + + return $last_in_nbs + if ( defined $charset + and defined $last_in_charset + and $charset eq $last_in_charset); + + my $nbs = "\xA0"; + my $length; + if (defined $charset and length $charset) + { + eval ("\$length = Encode::from_to(\$nbs, \"latin-1\", + \$charset, + 1)"); + } + # fall back solution + $nbs = "PO4A:VERY_IMPROBABLE_STRING_USEDFOR_NON-BREAKING-SPACES" + unless defined $length; + $last_in_charset = $charset; + $last_in_nbs = $nbs; + + return $last_in_nbs; + } + + my $last_out_charset; + my $last_out_nbs; +# get_out_nbs(charset) +# Return the representation of a non breaking space in the output charset +# (given in argument). +# or undef if this character doesn't exist in this charset. + sub get_out_nbs() { + my $charset = shift; + + return $last_out_nbs + if ( defined $charset + and defined $last_out_charset + and $charset eq $last_out_charset); + + my $nbs = "\xA0"; + my $length; + if (defined $charset and length $charset) + { + eval ("\$length = Encode::from_to(\$nbs, \"latin-1\", + \$charset, + 1)"); + } + # fall back solution + undef $nbs + unless defined $length; + $last_out_charset = $charset; + $last_out_nbs = $nbs; + + return $last_out_nbs; + } + +} + +# We can't push the header in the first line of the document, as in the +# other module, because the first line may contain indications on how the +# man page must be processed. +sub docheader { + return ""; +} + +# The header is pushed just before the .TH macro (this macro is mandatory +# and must be specified at the begining (there may be macro definitions +# before). +sub push_docheader { + my $self = shift; + $self->pushline( +".\\\"*******************************************************************\n". +".\\\"\n". +".\\\" This file was generated with po4a. Translate the source file.\n". +".\\\"\n". +".\\\"*******************************************************************\n" + ); +} + +# Split request's arguments. +# see: +# info groff --index-search "Request Arguments" +sub splitargs { + my ($ref,$arguments) = ($_[0],$_[1]); + my @args=(); + my $buffer=""; + my $escaped=0; + if (! defined $arguments) { + return @args; + } + # change non-breaking space before to ensure that split does what we want + # We change them back before pushing into the arguments. The one which + # will be translated will have the same change again (in pre_trans and + # post_trans), but the ones which won't get translated are not changed + # anymore. Let's play safe. + $arguments =~ s/\\ /$nbs/g; + $arguments =~ s/^ +//; + $arguments =~ s/\\&"/\\(dq/g; + $arguments =~ s/^ *//; + while (length $arguments) { + if ($arguments =~ s/^"((?:[^"]|"")*)"(?!") *//) { + my $a = $1; + $a =~ s/""/"/g if defined $a; + push @args,$a; + } elsif ($arguments =~ s/^"((?:[^"]|"")*) *$//) { + # Unterminated quote, but this seems to be handled by removing + # the trailing spaces and closing the quotes. + my $a = $1; + $a =~ s/""/"/g if defined $a; + push @args,$a; + } elsif ($arguments =~ s/^([^ ]+) *//) { + push @args,$1; + } else { + die wrap_ref_mod($ref, "po4a::man", dgettext("po4a", + "Cannot parse command arguments: %s"), + $arguments) + } + } + if ($debug{'splitargs'}) { + print STDERR "ARGS="; + map { print STDERR "$_^"} @args; + print STDERR "\n"; + } + + return @args; +} + +{ + #static variables + # font stack. + # Keep track of the current font (because a font modifier can + # stay open at the end of a paragraph), and the previous font (to + # handle \fP) + my $current_font = "R"; + my $previous_font = "R"; + # $regular_font describe the "Regular" font, which is the font used + # when there is no font modifier. + # For example, .SS use a Bold font, and thus in + # .SS This is a \fRsubsection\fB header + # the \fR and \fB font modifiers have to be kept. + my $regular_font = "R"; + + # Set the regular font + # It takes the regular font in argument (when no argument is provided, + # it uses "R"). + sub set_regular { + print STDERR "set_regular('@_')\n" + if ($debug{'fonts'}); + set_font(@_); + $regular_font = $current_font; + } + + sub set_font { + print STDERR "set_font('@_')\n" + if ($debug{'fonts'}); + my $saved_previous = $previous_font; + $previous_font = $current_font; + + if (! defined $_[0]) { + $current_font = "R"; + } elsif ($_[0] =~ /^(P|\[\]|\[P\])/) { + $current_font = $saved_previous; + } elsif (length($_[0]) == 1) { + $current_font = $_[0]; + } elsif (length($_[0]) == 2) { + $current_font = "($_[0]"; + } else { + $current_font = "[$_[0]]"; + } + print STDERR "r:'$regular_font', p:'$previous_font', c:'$current_font'\n" + if ($debug{'fonts'}); + } + + sub do_fonts { + # one argument: a string + my ($str, $ref) = (shift, shift); + print STDERR "do_fonts('$str', '$ref')=" + if ($debug{'fonts'}); + + # restore the font stack + $str = "\\f$previous_font\\f$current_font".$str; + # In order to be able to split on /\\f/, without problem with + # \\foo, groff backslash (\\) are changed to the (equivalent) + # form: \e (this should be done in shiftline). + my @array1=split(/\\f/, $str); + + $str = shift @array1; # The first element is always empty because + # the $current_font was put at the beginning + # $last_font indicates the last font that was appended to the buffer. + # It differ from $current_font because concecutive identical fonts + # are not written in the buffer. + my $last_font=$regular_font; + + foreach my $elem (@array1) { + # Do not touch the fonts in the inline macros + # These inline macros may have their argument in bold or italic, + # we can't know. + if ($str =~ m/E<\.([^>]|E|E)*$/s) { + # We can't use \\f here, otherwise the font simplifier regexp + # will use the fonts of the inline macros. + $str .= "PO4A-FAKE-FONT".$elem; + next; + } + + # Replace \fP by the exact font (because some font modifiers will + # be removed or added, which will break groff's font stack) + $elem =~ s/^(P|\[\]|\[P\])/$previous_font/s; + # change \f1 to \fR, etc. + # Those fonts are defined in the DESC file, which + # may depend on the groff device. + # fonts 1 to 4 are usually mapped to R, I, B, BI + # TODO: use an array for the font positions. This + # array should be updated by .fp requests. + $elem =~ s/^1/R/; + $elem =~ s/^2/I/; + $elem =~ s/^3/B/; + $elem =~ s/^4/(BI/; + + if ($elem =~ /^([1-4]|B|I|R|\(..|\[[^]]*\]|L)(.*)$/s) { + # Each element should now start by a recognized font modifier + my $new_font = $1; + my $arg = $2; + # Update the font stack + $previous_font = $current_font; + $current_font = $new_font; + + if ($new_font eq $last_font) { + # continue with the same font. + $str.=$arg; + } else { + # A new font is used, update $last_font + $last_font = $new_font; + $str .= "\\f".$elem; + } + } else { + die wrap_ref_mod($ref, + "po4a::man", + dgettext("po4a", + "Unsupported font in: '%s'."), + "\\f".$elem); + } + } + # Do some simplification (they don't change the font stack) + # Remove empty font modifiers at the end + $str =~ s/($FONT_RE)*$//s; + + # close any font modifier + if ($str =~ /.*($FONT_RE)(.*?)$/s && $1 ne "\\f$regular_font") { + $str =~ s/(\n?)$/\\f$regular_font$1/; + } + + # remove fonts with empty argument + while ($str =~ /($FONT_RE){2}/) { + # while $str has two consecutive font modifiers + # only keep the second one. + $str =~ s/($FONT_RE)($FONT_RE)/$2/s; + } + + # when there are two consecutive switches to the regular font, + # remove the last one. + while ($str =~ /^(.*)\\f$regular_font # anything followed by a + # regular font + ((?:\\(?!f)|[^\\])*) # the text concerned by + # this font (i.e. without any + # font modifier, i.e. it + # contains no '\' followed by + # an 'f') + \\f$regular_font # another regular font + (.*)$/sx) { + $str = "$1\\f$regular_font$2$3"; + } + + # the regular font modifier at the beginning of the string is not + # needed (the do_fonts subroutine ensure that every paragraph ends with + # the regular font. + if ($str =~ /^(.*?)\\f$regular_font(.*)$/s && $1 !~ /$FONT_RE/) { + $str = "$1$2"; + } + + # Use special markup for common fonts, so that translators don't see + # groff's font modifiers + my $PO_FONTS = "B|I|R|\\(CW"; + # remove the regular font from this list + $PO_FONTS =~ s/^$regular_font\|//; + $PO_FONTS =~ s/\|$regular_font\|/|/; + $PO_FONTS =~ s/\|$regular_font$//; + while ($str =~ /^(.*?) # $1: anything (non greedy: as + # few as possible) + \\f($PO_FONTS) # followed by a common font + # modifier ($2) + ((?:\\[^f]|[^\\])*) # $3: the text concerned by + # this font (i.e. without any + # font modifier, i.e. it + # contains no '\' followed by + # an 'f') + \\f # the next font modifier + (.*)$/sx) { # $4: anything up to the end + my ($begin, $font, $arg, $end) = ($1,$2,$3,$4); + if ($end =~ /^$regular_font(.*)$/s) { + # no need to add a switch to $regular_font + $str = $begin."$font<$arg>$1"; + } else { + $str = $begin."$font<$arg>\\f$end"; + } + } + $str =~ s/\(CW> +# is passed (".TH","LS","1","October 2002","ls (coreutils) 4.5.2","User Commands") +# Macro name is also passed, because .B (bold) will be encoded in pod format (and mangeled). +# They should return a list, which will be join'ed(' ',..) +# or undef when they don't want to add anything + +# Some well known macro handling + +# For macro taking only one argument, but people may forget the quotes. +# Example: >>.SH Another Section<< which should be >>.SH "Another Section"<< +sub translate_joined { + my ($self,$macroname,$macroarg)=(shift,shift,join(" ",@_)); + #section# .S[HS] name + + $self->pushmacro($macroname, + $self->t($macroarg)); +} + +# For macro taking several arguments, having to be translated separately +sub translate_each { + my ($self,$first)= (shift,0); + $self->pushmacro( map { $first++ ?$self->t($_):$_ } @_); +} + +# For macro which shouldn't be given any arg +sub noarg { + my $self = shift; + warn "Macro $_[0] does not accept any argument\n" + if (defined ($_[1])); + $self->pushmacro(@_); +} + +# For macro whose arguments shouldn't be translated +sub untranslated { + my ($self,$first)= (shift,0); + $self->pushmacro( map { $first++ ?$self->r($_):$_ } @_); +} + +### +### man 7 man +### + +$macro{'TH'}= sub { + my $self=shift; + my ($th,$title,$section,$date,$source,$manual)=@_; + #Preamble#.TH title section date source manual +# print STDERR "TH=$th;titre=$title;sec=$section;date=$date;source=$source;manual=$manual\n"; + + # Reset the memories + $self->push_docheader(); + + $self->pushmacro($th, + $self->t($title), + $section, + $self->t($date), + $self->t($source), + $self->t($manual)); +}; + +# .SS t Subheading t (like .SH, but used for a subsection inside a section). +$macro{'SS'}=$macro{'SH'}=sub { + if (!defined $_[2]) { + # The argument is on the next line. + my ($self,$macroname) = (shift,shift); + my ($l2,$ref2) = $self->shiftline(); + if ($l2 =~/^\./) { + $self->SUPER::unshiftline($l2,$ref2); + } else { + chomp($l2); + $self->pushmacro($macroname, + $self->t($l2)); + } + return; + } else { + return translate_joined(@_); + } +}; + +# Macro: .SM [text] +# Set the text on the same line or the text on the next line in a +# font that is one point size smaller than the default font. +# FIXME: Maybe we should find a better way to represent this (inline is +# not really nice in the PO). +$inline{'SM'}=1; + +# .SP n Skip n lines (I think) +$macro{'SP'}=\&untranslated; + +#Normal Paragraphs +# .LP Same as .PP (begin a new paragraph). +# .P Same as .PP (begin a new paragraph). +# .PP Begin a new paragraph and reset prevailing indent. +#Relative Margin Indent +# .RS i Start relative margin indent - moves the left margin i to the right +# As a result, all following paragraph(s) will be indented until +# the corresponding .RE. +# .RE End relative margin indent. +$macro{'LP'}=$macro{'P'}=$macro{'PP'}=sub { + noarg(@_); + + # From info groff: + # The font size and shape are reset to the default value (10pt roman if no + # `-rS' option is given on the command line). + set_font("R"); +}; +$macro{'RE'}=\&noarg; +$macro{'RS'}=\&untranslated; + +sub parse_tp_tq { + my $self=shift; + my ($line,$l2,$ref2); + $line .= $_[0] if defined($_[0]); + $line .= ' '.$_[1] if defined($_[1]); + $self->pushline($self->r($line)."\n"); + + ($l2,$ref2) = $self->shiftline(); + chomp($l2); + while ($l2 =~ /^\.PD/) { + $self->pushline($self->r($l2)."\n"); + ($l2,$ref2) = $self->shiftline(); + chomp($l2); + } + if ($l2 =~/^([.'][\t ]*([^\t ]*))(?:([\t ]+)(.*)$|$)/) { + if ($inline{$2}) { + my $tmp = ""; + if (defined $4 and length $4) { + $tmp = $3.$self->t($4, "wrap" => 0); + } + $self->pushline($1.$tmp."\n"); + } else { + # If the line after a .TP is a macro, + # let the parser do it's job. + # Note: use Transtractor unshiftline for now. This may require an + # implementation of the man module's own implementation. + # This may be a problem if, for example, the line resulted + # of a line continuation. + $self->SUPER::unshiftline($l2,$ref2); + } + } else { + $self->pushline($self->t($l2, "wrap" => 0)."\n"); + } +} + +#Indented Paragraph Macros +# .TP i Begin paragraph with hanging tag. The tag is given on the next line, +# but its results are like those of the .IP command. +$macro{'TP'}=sub { + parse_tp_tq(@_); + + # From info groff: + # Note that neither font shape nor font size of the label [i.e. argument + # or first line] is set to a default value; on the other hand, the rest of + # the text has default font settings. + set_font("R"); +}; + +# Indented Paragraph Macros +# .TQ Indicates continuation of the .TP labels that precede the indented +# paragraph. +$macro{'TQ'}=sub { + warn "Macro $_[1] does not accept any argument\n" + if (defined ($_[2])); + + parse_tp_tq(@_); +}; + +# Indented Paragraph Macros +# .HP i Begin paragraph with a hanging indent (the first line of the paragraph +# is at the left margin of normal paragraphs, and the rest of the para- +# graph's lines are indented). +# +$macro{'HP'}=sub { + untranslated(@_); + + # From info groff: + # Font size and face are reset to their default values. + set_font("R"); +}; + +# Indented Paragraph Macros +# .IP [designator] [nnn] +# Sets up an indented paragraph, using designator as a tag to mark +# its beginning. The indentation is set to nnn if that argument is +# supplied (default unit is `n'), otherwise the default indentation +# value is used. Font size and face of the paragraph (but not the +# designator) are reset to its default values. To start an indented +# paragraph with a particular indentation but without a designator, +# use `""' (two doublequotes) as the second argument. + +# Note that the above is the groff_man(7) version, which of course differs radically +# from man(7). In one case, the designator is optional and the nnn is not, and the +# contrary in the other. This implies that when sticking to groff_man(7), we should +# mark an uniq argument as translatable. + +$macro{'IP'}=sub { + my $self=shift; + if (defined $_[2]) { + $self->pushmacro($_[0],$self->t($_[1]),$_[2]); + } elsif (defined $_[1]) { + $self->pushmacro($_[0],$self->t($_[1])); + } else { + $self->pushmacro(@_); + } + + # From info groff: + # Font size and face of the paragraph (but not the designator) are reset + # to their default values. + set_font("R"); +}; + +# Hypertext Link Macros +# .UR u Begins a hypertext link to the URI (URL) u; it will end with +# the corresponding UE command. When generating HTML this should +# translate into the HTML command . +# There is an exception: if u is the special value ":", then no +# hypertext link of any kind will be generated until after the +# closing UE (this permits disabling hypertext links in +# phrases like LALR(1) when linking is not appropriate). +# .UE Ends the corresponding UR command; when generating HTML this +# should translate into . +# .UN u Creates a named hypertext location named u; do not include a +# corresponding UE command. +# When generating HTML this should translate into the HTML command +#   +$macro{'UR'}=sub { + return untranslated(@_) + if (defined($_[2]) && $_[2] eq ':'); + return translate_joined(@_); +}; +$macro{'UE'}=\&noarg; +$macro{'UN'}=\&translate_joined; + +# Miscellaneous Macros +# .DT Reset tabs to default tab values (every 0.5 inches); does not +# cause a break. +# .PD d Set inter-paragraph vertical distance to d (if omitted, d=0.4v); +# does not cause a break. +$macro{'DT'}=\&noarg; +$macro{'PD'}=\&untranslated; + +# Indexing term (printed on standard error). +# (ms macros) +$macro{'IX'}=\&translate_each; + +### +### groff macros +### +# .br +$macro{'br'}=\&noarg; +# .bp N Eject current page and begin new page. +$macro{'bp'}=\&untranslated; +# .ad Begin line adjustment for output lines in current adjust mode. +# .ad c Start line adjustment in mode c (c=l,r,b,n). +$macro{'ad'}=\&untranslated; +# .de macro Define or redefine macro until .. is encountered. +$macro{'de'}=sub { + my $self = shift; + if ($groff_code ne "fail") { + my $paragraph = "@_"; + my $end = "."; + if ($paragraph=~/^[.'][\t ]*de[\t ]+([^\t ]+)[\t ]+([^\t ]+)[\t ]$/) { + $end = $2; + } + my ($line, $ref) = $self->SUPER::shiftline(); + chomp $line; + $paragraph .= "\n".$line; + while (defined($line) and $line ne ".$end") { + ($line, $ref) = $self->SUPER::shiftline(); + if (defined $line) { + chomp $line; + $paragraph .= "\n".$line; + } + } + $paragraph .= "\n"; + if ($groff_code eq "verbatim") { + $self->pushline( $self->r($paragraph) ); + } else { + $self->pushline( $self->translate($paragraph, + $self->{ref}, + "groff code", + "wrap" => 0) ); + } + } else { + die wrap_ref_mod($self->{ref}, "po4a::man", dgettext("po4a", "This page defines a new macro with '.de'. Since po4a is not a real groff parser, this is not supported.")); + } +}; +# .ds stringvar anything +# Set stringvar to anything. +$macro{'ds'}=sub { + my ($self, $m) = (shift,shift); + my $name = shift; + my $string = "@_"; + # indicate to which variable this corresponds. The translator can + # find references to this string in the translation "\*(name" or + # "\*[name]" + $self->{type} = "ds $name"; + $self->pushline($m." ".$self->r($name)." ".$self->translate($string)."\n"); +}; +# .fam Return to previous font family. +# .fam name Set the current font family to name. +$macro{'fam'}=\&untranslated; +# .fc a b Set field delimiter to a and pad character to b. +$macro{'fc'}=\&untranslated; +# .ft font Change to font name or number font; +$macro{'ft'}=sub { + if (defined $_[2]) { + set_font($_[2]); + } else { + set_font("P"); + } +}; +# .hc c Set up additional hyphenation indicator character c. +$macro{'hc'}=\&untranslated; +# .hy Enable hyphenation (see nh) +# .hy N Switch to hyphenation mode N. +# .hym n Set the hyphenation margin to n (default scaling indicator m). +# .hys n Set the hyphenation space to n. +$macro{'hy'}=$macro{'hym'}=$macro{'hys'}=\&untranslated; + +# .ie cond anything If cond then anything else goto .el. +# .if cond anything If cond then anything; otherwise do nothing. +$macro{'ie'}=$macro{'if'}=sub { + my $self = shift; + if ($groff_code ne "fail") { + my $m = $_[0]; + my $paragraph = "@_"; + my ($line,$ref); + my $count = 0; + $count = 1 if ($paragraph =~ m/(? 0)) { + ($line,$ref)=$self->SUPER::shiftline(); + chomp $line; + $paragraph .= "\n".$line; + $count += 1 if ($line =~ m/(?SUPER::shiftline(); + chomp $line; + while ($line =~ m/^[.']\\"/) { + $paragraph .= "\n".$line; + ($line,$ref)=$self->SUPER::shiftline(); + chomp $line; + } + + if ($line !~ m/^[.'][ \t]*el(\s|\\\{)/) { + die wrap_ref_mod($self->{ref}, "po4a::man", dgettext("po4a", + "The .ie macro must be followed by a .el macro.")); + } + my $paragraph2 = $line; + $count = 0; + $count = 1 if ($line =~ m/(? 0)) { + ($line,$ref)=$self->SUPER::shiftline(); + chomp $line; + $paragraph2 .= "\n".$line; + $count += 1 if ($line =~ m/(?pushline( $self->r($paragraph) ); + } else { + $self->pushline( $self->translate($paragraph, + $self->{ref}, + "groff code", + "wrap" => 0) ); + } + } else { + die wrap_ref_mod($self->{ref}, "po4a::man", dgettext("po4a", + "This page uses conditionals with '%s'. Since po4a is not a real groff parser, this is not supported."), $_[0]); + } +}; +# .in N Change indent according to N (default scaling indicator m). +$macro{'in'}=\&untranslated; + +# .ig end Ignore text until .end. +$macro{'ig'}=sub { + my $self = shift; + $self->pushmacro(@_); + my ($name,$end) = (shift,shift||''); + $end='' if ($end =~ m/^\\\"/); + my ($line,$ref)=$self->shiftline(); + while (defined($line)) { + $self->pushline($self->r($line)); + last if ($line =~ /^\.$end\./); + ($line,$ref)=$self->shiftline(); + } +}; + + +# .lf N file Set input line number to N and filename to file. +$macro{'lf'}=\&untranslated; +# .ll N Set line length according to N +$macro{'ll'}=\&untranslated; + +# .nh disable hyphenation (see hy) +$macro{'nh'}=\&untranslated; +# .na No Adjusting (see ad) +$macro{'na'}=\&untranslated; +# .ne N Need N vertical space +$macro{'ne'}=\&untranslated; +# .nr register N M +# Define or modify register +$macro{'nr'}=\&untranslated; +# .ps N Point size; same as \s[N] +$macro{'ps'}=\&untranslated; +# .so filename Include source file. +# .mso groff variant of .so (other search path) +$macro{'so'}= $macro{'mso'} = sub { + warn wrap_mod("po4a::man", dgettext("po4a", + "This page includes another file with '%s'. Do not forget to translate this file ('%s')."), $_[1], $_[2]); + my $self = shift; + $self->pushmacro(@_); +}; +# .sp Skip one line vertically. +# .sp N Space vertical distance N +$macro{'sp'}=\&untranslated; +# .vs [space] +# .vs +space +# .vs -space +# Change (increase, decrease) the vertical spacing by SPACE. The +# default scaling indicator is `p'. +$macro{'vs'}=\&untranslated; +# .ta T N Set tabs after every position that is a multiple of N. +# .ta n1 n2 ... nn T r1 r2 ... rn +# Set tabs at positions n1, n2, ..., nn, [...] +$macro{'ta'}=sub { + # In some cases, a ta request can contain a translatable argument. + # FIXME: detect those cases (something like 5i does not need to be + # translated) + my ($self,$m)=(shift,shift); + my $line = "@_"; + $line =~ s/^ +//; + $self->pushline($m." ".$self->translate($line,$self->{ref},'ta')."\n"); +}; +# .ti +N Temporary indent next line (default scaling indicator m). +$macro{'ti'}=\&untranslated; + + +### +### tbl macros +### +$macro{'TS'}=sub { + my $self=shift; + my ($in_headers,$buffer)=(1,""); + my ($line,$ref)=$self->shiftline(); + + # Push table start + $self->pushmacro(@_); + while (defined($line)) { + if ($line =~ /^\.TE/) { + # Table end + $self->pushline($self->r($line)); + return; + } + if ($in_headers) { + if ($line =~ /\.$/) { + $in_headers = 0; + } + $self->pushline($self->r($line)); + } elsif ($line =~ /\\$/) { + # Lines are continued on \ at the end of line + $buffer .= $line; + } else { + $buffer .= $line; + # Arguments to translate are separated by \t + $self->pushline(join("\t", + map { $self->translate($buffer, + $ref, + 'tbl table') + } split (/\\t/,$line))); + $buffer = ""; + } + ($line,$ref)=$self->shiftline(); + } +}; + +### +### info groff +### + +## Builtin register, of course they do not need to be translated + +$macro{'F'}=$macro{'H'}=$macro{'V'}=$macro{'A'}=$macro{'T'}=\&untranslated; + +## ms package +## +# +# Displays and keeps. None of these macro accept a translated argument +# (they allow to make blocks of text which cannot be broken by new page) + +$macro{'DS'}=$macro{'LD'}=$macro{'DE'}=\&untranslated; +$macro{'ID'}=$macro{'BD'}=$macro{'CD'}=\&untranslated; +$macro{'RD'}=$macro{'KS'}=$macro{'KE'}=\&untranslated; +$macro{'KF'}=$macro{'B1'}=$macro{'B2'}=\&untranslated; +$macro{'DA'}=\&translate_joined; + +# .pc c Change page number character +$macro{'pc'}=\&translate_joined; + +# .ns Disable .sp and such +# .rs Enable them again +$macro{'ns'}=$macro{'rs'}=\&untranslated; + +# .cs font [width [em-size]] +# Switch to and from "constant glyph space mode". +$macro{'cs'}=\&untranslated; + +# .ss word_space_size [sentence_space_size] +# Change the minimum size of a space between filled words. +$macro{'ss'}=\&untranslated; + +# .ce Center one line horizontally +# .ce N Center N lines +# .ul N Underline N lines (but not the spaces) +# .cu N Underline N lines (even the spaces) +$macro{'ce'}=$macro{'ul'}=$macro{'cu'}=sub { + my $self=shift; + if (defined $_[1]) { + if ($_[1] <= 0) { + # disable centering, underlining, ... + $self->pushmacro($_[0]); + } else { +# All of these are not handled yet because the number of line may change +# during the translation + die wrap_mod("po4a::man", dgettext("po4a", + "This page uses the '%s' request with the number of lines in argument. This is not supported yet."), $_[0]); + } + } else { + $self->pushmacro($_[0]); + } +}; + +# .ec [c] +# Set the escape character to C. With no argument the default +# escape character `\' is restored. It can be also used to +# re-enable the escape mechanism after an `eo' request. +$macro{'ec'}=sub { + my $self=shift; + if (defined $_[1]) { + die wrap_mod("po4a::man", dgettext("po4a", + "This page uses the '%s' request. This request is only supported when no argument is provided."), $_[0]); + } else { + $self->pushmacro($_[0]); + } +}; + + +### +### BSD compatibility macros: .AT and .UC +### (define the version of Berkley used) +### FIXME: the header ("3rd Berkeley Distribution" or such) declared +### by this macro isn't translatable we may want to remove +### this from the generated manpage, and declare our own header +### +$macro{'UC'}=$macro{'AT'}=\&untranslated; + +# Request: .hw word1 word2 ... +# Define how WORD1, WORD2, etc. are to be hyphenated. The words +# must be given with hyphens at the hyphenation points. +# +# If the English page needs to specify how a word must be hyphenated, the +# translated page may also have this need. +$macro{'hw'}=\&translate_each; + + +############################################################################# +# +# mdoc macros +# +# The macros are defined in mdoc(7) and groff_mdoc(7) +# +# TBC: Should the font processing be disabled in the mdoc mode? +############################################################################# +# FIXME: Maybe we should verify that the page is an mdoc page +# (add a flag in Dd, and always check that this flag is set in the +# other mdoc macros) +sub translate_mdoc { + my ($self,$macroname)=(shift,shift); + my $macroarg = ""; + foreach (@_) { + $macroarg.=" " if (length $macroarg); + if ($_ =~ m/((?pushline("$macroname ".$self->t($macroarg)."\n"); +} +sub translate_mdoc_no_quotes { + my ($self,$macroname, $macroarg)=(shift,shift, join(" ", @_)); + + $self->pushline("$macroname ".$self->t($macroarg)."\n"); +} +# +# Title Macros +# ============ +# .Dd Month day, year Document date. +$macro{'Dd'}=sub { + my ($self,$macroname,$macroarg)=(shift,shift,join(" ",@_)); + + $mdoc_mode = 1; + $self->push_docheader(); + +# FIXME: It would be nice if we could switch from one set of macros to the +# other. +# +# This does not work at this time. If we erase the current set of macros, +# po4a fails when a configuration file uses both mdoc and groff pages. +# +# # Erase the current macro definitions +# %macro=(); +# %inline=(); +# %no_wrap_begin=(); +# %no_wrap_end=(); + # Use the mdoc macros + define_mdoc_macros(); + + $self->translate_mdoc_no_quotes($macroname,$macroarg); +}; + +sub define_mdoc_macros { + # .Dt DOCUMENT_TITLE [section] [volume] Title, in upper case. + $macro{'Dt'}=\&translate_mdoc; + # .Os OPERATING_SYSTEM [version/release] Operating system (BSD). + $macro{'Os'}=\&translate_each; + # Keep the quotes e.g. finger.1 + # Don't add quotes e.g. logger.1 + + # Page Layout Macros + # ================== + # .Sh Section Headers. + # (man mdoc indicates only a limited set of valid headers, + # but it should be OK to translate the header) + $macro{'Sh'}= sub { + my ($self,$macroname)=(shift,shift); + my $macroarg = ""; + foreach (@_) { + $macroarg.=" " if (length $macroarg); + if ($_ =~ m/((?pushline("$macroname ".$self->r($macroarg)."\n"); + } else { + $self->pushline("$macroname ".$self->t($macroarg)."\n"); + } + }; + # .Ss Subsection Headers. + $macro{'Ss'}=\&translate_mdoc; + # .Pp Paragraph Break. Vertical space (one line). + $macro{'Pp'}=\&noarg; + # .Lp Same as .Pp + $macro{'Lp'}=\&noarg; + # .D1 (D-one) Display-one Indent and display one text line. + $macro{'D1'}=\&translate_mdoc; + # .Dl (D-ell) Display-one literal. + # Indent and display one line of literal text + $macro{'Dl'}=\&translate_mdoc; + # .Bd Begin-display block. + # FIXME: Note: there are some options, some of the options argument + # may be translatable (-file , -offset ) + $no_wrap_begin{'Bd'} = 1; + # .Ed End-display (matches .Bd). + $no_wrap_end{'Ed'} = 1; + # .Bl Begin-list. Create lists or columns. + # FIXME: As for .Bd, there are some options + $macro{'Bl'}=\&untranslated; + # .El End-list. + $macro{'El'}=\&noarg; + # .It List item. + # FIXME: Maybe we could extract other modifiers + # as in .It Fl l Ar num + $macro{'It'}=\&translate_mdoc; + # .Lk html link + $macro{'Lk'}=\&untranslated; + + # Manual Domain Macros + # ==================== + # FIXME: I think most Manual and General text domain are in the inline category + foreach (qw(Ad An Ar Cd Cm Dv Er Ev Fa Fd Fn Ic Li Nm Op Ot Pa St Va Vt Xr)) { + $inline{$_} = 1; + } + # FIXME: some of these macros introduce a line in bold. + # Using \fP in these line is not supported. + # do_fonts should be called for every inline line + + # General Text Domain + # =================== + foreach (qw(%A %B %C %D %I %J %N %O %P %Q %R %T %U %V + Ac Ao Ap Aq At Bc Bf Bo Bq Brc Bro Brq Bx Db Dc Do Dq Ec Ef Em Eo Eq Fx No Ns + Pc Pf Po Pq Qc Ql Qo Qq Re Rs Rv Sc So Sq Sm Sx Sy Tn Ux Xc Xo)) { + $inline{$_} = 1; + } + + # FIXME: Maybe it should be joined with the preceding .Nm + $macro{'Nd'}=\&translate_mdoc; + + # Command line flags + $inline{'Fl'} = 1; + # Exit status + $inline{'Ex'} = 1; + # Opening option bracket + $inline{'Oo'} = 1; + # Closing option bracket + $inline{'Oc'} = 1; + # Begin keep (keep words in the same line) + $inline{'Bk'} = 1; + # End keep + $inline{'Ek'} = 1; + # Library Names + $inline{'Lb'} = 1; + # Function Types + $inline{'Ft'} = 1; + # Function open (for functions with many arguments) + $inline{'Fo'} = 1; + # Function close + $inline{'Fc'} = 1; + # OpenBSD macro + $inline{'Ox'} = 1; + # BSD/OS Macro + $inline{'Bsx'} = 1; + # #include statements + $macro{'In'} = \&translate_mdoc; + # NetBSD Macro + $inline{'Nx'} = 1; + # Curly brackets + $inline{'Brq'} = 1; + # Corporate name + $inline{'%Q'} = 1; + # Math symbol + $inline{'Ms'} = 1; + # Prints 'under development' + $inline{'Ud'} = 1; + + # This macro is a groff macro. I don't know if ot is valid in an mdoc page. + # But this is used in some pages and seems to work + $macro{'br'}=\&noarg; + +} # end of define_mdoc_macros Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Texinfo.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Texinfo.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Texinfo.pm (revision 2840) @@ -0,0 +1,545 @@ +#!/usr/bin/perl -w + +# Copyright (c) 2004-2007 by Nicolas FRANÇOIS +# +# This file is part of po4a. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with po4a; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Texinfo - convert Texinfo documents and derivates from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Texinfo is a module to help the translation of Texinfo documents into +other [human] languages. + +This module contains the definitions of common Texinfo commands and +environments. + +=head1 STATUS OF THIS MODULE + +This module is still beta. +Please send feedback and feature requests. + +=head1 SEE ALSO + +L, +L, +L + +=head1 AUTHORS + + Nicolas François + +=head1 COPYRIGHT AND LICENSE + +Copyright 2004-2007 by Nicolas FRANÇOIS . + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see COPYING file). + +=cut + +package Locale::Po4a::Texinfo; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw($VERSION @ISA @EXPORT); +$VERSION= $Locale::Po4a::TeX::VERSION; +@ISA= qw(Locale::Po4a::TeX); +@EXPORT= qw(); + +use Locale::Po4a::Common; +use Locale::Po4a::TeX; +use subs qw( + &parse_definition_file + ®ister_generic_command &is_closed &translate_buffer + ®ister_verbatim_environment + &generic_command + &in_verbatim); +*parse_definition_file = \&Locale::Po4a::TeX::parse_definition_file; +*register_generic_command = \&Locale::Po4a::TeX::register_generic_command; +*register_verbatim_environment = \&Locale::Po4a::TeX::register_verbatim_environment; +*generic_command = \&Locale::Po4a::TeX::generic_command; +*is_closed = \&Locale::Po4a::TeX::is_closed; +*in_verbatim = \&Locale::Po4a::TeX::in_verbatim; +*translate_buffer = \&Locale::Po4a::TeX::translate_buffer; +use vars qw($RE_ESCAPE $ESCAPE + $RE_VERBATIM + $RE_COMMENT $RE_PRE_COMMENT + $no_wrap_environments $separated_commands + %commands %environments + %command_categories %separated + %env_separators %debug + %translate_buffer_env + @exclude_include @comments); +*RE_ESCAPE = \$Locale::Po4a::TeX::RE_ESCAPE; +*ESCAPE = \$Locale::Po4a::TeX::ESCAPE; +*RE_VERBATIM = \$Locale::Po4a::TeX::RE_VERBATIM; +*RE_COMMENT = \$Locale::Po4a::TeX::RE_COMMENT; +*RE_PRE_COMMENT = \$Locale::Po4a::TeX::RE_PRE_COMMENT; +*no_wrap_environments = \$Locale::Po4a::TeX::no_wrap_environments; +*separated_commands = \$Locale::Po4a::TeX::separated_commands; +*commands = \%Locale::Po4a::TeX::commands; +*environments = \%Locale::Po4a::TeX::environments; +*command_categories = \%Locale::Po4a::TeX::command_categories; +*separated = \%Locale::Po4a::TeX::separated; +*env_separators = \%Locale::Po4a::TeX::env_separators; +*debug = \%Locale::Po4a::TeX::debug; +*translate_buffer_env = \%Locale::Po4a::TeX::translate_buffer_env; +*exclude_include = \@Locale::Po4a::TeX::exclude_include; +*comments = \@Locale::Po4a::TeX::comments; + +$ESCAPE = "\@"; +$RE_ESCAPE = "\@"; +$RE_VERBATIM = "\@example"; +$RE_COMMENT = "\\\@(?:c|comment)\\b"; +$RE_PRE_COMMENT = "(?pushline(<{type}; + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { + chomp($line); + $self->{ref}="$ref"; + + if ($line =~ /^\s*@\s*po4a\s*:/) { + parse_definition_line($self, $line); + goto LINE; + } + + my $closed = 1; + if (!in_verbatim(@env)) { + $closed = is_closed($paragraph); + } +# if (not $closed) { +# print "not closed. line: '$line'\n para: '$paragraph'\n"; +# } + + if ($closed and $line =~ /^\s*$/) { + # An empty line. This indicates the end of the current + # paragraph. + $paragraph .= $line."\n"; + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + } + } elsif ($line =~ m/^\\input /) { + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + } + $self->pushline($line."\n"); + $self->push_docheader(); + } elsif ($line =~ m/^$RE_COMMENT/) { + $self->push_docheader(); + $self->pushline($line."\n"); + } elsif ( $closed + and ($line =~ /^@([^ ]*?)(?: +(.*))?$/) + and (defined $commands{$1}) + and ($break_line{$1})) { + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + } + my $arg = $2; + my @args = (); + if (defined $arg and length $arg) { + # FIXME: keep the spaces ? + $arg =~ s/\s*$//s; + @args= (" ", $arg); + } + ($t, @env) = &{$commands{$1}}($self, $1, "", \@args, \@env, 1); + $self->pushline($t."\n"); + } else { + # continue the same paragraph + $paragraph .= $line."\n"; + } + + # Reinit the loop + ($line,$ref)=$self->shiftline(); + undef $self->{type}; + } + + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + } +} # end of parse + +sub line_command { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "line_command($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'}); + + my $translated = $ESCAPE.$command; + my $line = $args->[1]; + if (defined $line and length $line) { + if ( defined $translate_line_command{$command} + and $translate_line_command{$command}) { + # $no_wrap could be forced to 1, but it should already be set + my ($t,$e) = $self->translate_buffer($line,$no_wrap,@$env,$command); + $translated .= " ".$t; + } else { + $translated .= " ".$line; + } + } + print "($translated,@$env)\n" + if ($debug{'commands'}); + return ($translated,@$env); +} + +sub defindex_line_command { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "line_command($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'}); + my $idx = $$args[1]."index"; + $commands{$idx} = \&line_command; + $break_line{$idx} = 1; + $translate_line_command{$idx} = 1; + + return line_command($self,$command,$variant,$args,$env,$no_wrap); +} + +sub translate_buffer_menu { + my ($self,$buffer,$no_wrap,@env) = (shift,shift,shift,@_); + print STDERR "translate_buffer_menu($buffer,$no_wrap,@env)=" + if ($debug{'translate_buffer'}); + + my $translated_buffer = ""; + my $spaces = ""; + if ($buffer =~ m/(\s*)$/s) { + $spaces = $1; + } + + + while ($buffer =~ m/^(.*?)((?:\n|^)\* )(.*)$/s) { + my $sep = $2; + $buffer = $3; + my($t, @e) = $self->translate_buffer_menuentry($1, $no_wrap, + @env, "menuentry"); + $translated_buffer .= $t.$sep; + } + my($t, @e) = $self->translate_buffer_menuentry($buffer, $no_wrap, + @env, "menuentry"); + $translated_buffer .= $t; + + $translated_buffer .= $spaces; + + print STDERR "($translated_buffer,@env)\n" + if ($debug{'translate_buffer'}); + return ($translated_buffer,@env); +} +$translate_buffer_env{"menu"} = \&translate_buffer_menu; +$translate_buffer_env{"detailmenu"} = \&translate_buffer_menu; +$translate_buffer_env{"direntry"} = \&translate_buffer_menu; + +my $menu_width = 78; +my $menu_sep_width = 30; +sub translate_buffer_menuentry { + my ($self,$buffer,$no_wrap,@env) = (shift,shift,shift,@_); + print STDERR "translate_buffer_menuentry($buffer,$no_wrap,@env)=" + if ($debug{'translate_buffer'}); + + my $translated_buffer = ""; + + if ( $buffer =~ m/^(.*?)(::)\s+(.*)$/s + or $buffer =~ m/^(.*?: .*?)(\.)\s+(.*)$/s) { + my ($name, $sep, $description) = ($1, $2, $3); + my ($t, @e) = $self->translate_buffer($name, $no_wrap, @env); + $translated_buffer = $t.$sep." "; + my $l = length($translated_buffer) + 2; + if ($l < $menu_sep_width-1) { + $translated_buffer .= ' 'x($menu_sep_width-1-$l); + $l = $menu_sep_width-1; + } + ($t, @e) = $self->translate_buffer($description, $no_wrap, @env); + $t =~ s/\n//sg; + $t = Locale::Po4a::Po::wrap($t, $menu_width-$l-2); + my $spaces = ' 'x($l+2); + $t =~ s/\n/\n$spaces/sg; + $translated_buffer .= $t; + } else { +# FIXME: no-wrap if a line start by a space + my ($t, @e) = $self->translate_buffer($buffer, $no_wrap, @env); + $translated_buffer = $t; + } + + print STDERR "($translated_buffer,@env)\n" + if ($debug{'translate_buffer'}); + return ($translated_buffer,@env); +} + +sub translate_buffer_ignore { + my ($self,$buffer,$no_wrap,@env) = (shift,shift,shift,@_); + print STDERR "translate_buffer_ignore($buffer,$no_wrap,@env);\n" + if ($debug{'translate_buffer'}); + return ($buffer,@env); +} +$translate_buffer_env{"ignore"} = \&translate_buffer_ignore; + +foreach (qw(appendix section cindex findex kindex opindex pindex vindex subsection + dircategory subtitle include + exdent center unnumberedsec + heading unnumbered unnumberedsubsec + unnumberedsubsubsec appendixsec appendixsubsec + appendixsubsubsec majorheading chapheading subheading + subsubheading shorttitlepage + subsubsection top item itemx chapter settitle + title author)) { + $commands{$_} = \&line_command; + $break_line{$_} = 1; + $translate_line_command{$_} = 1; +} +foreach (qw(c comment clear set setfilename setchapternewpage vskip synindex + syncodeindex need fonttextsize printindex headings finalout sp + definfoenclose)) { + $commands{$_} = \&line_command; + $break_line{$_} = 1; +} +foreach (qw(defcodeindex defindex)) { + $commands{$_} = \&defindex_line_command; + $break_line{$_} = 1; +} +# definfoenclose: command definition => translate? +foreach (qw(insertcopying page bye summarycontents shortcontents contents + noindent)) { + $commands{$_} = \&line_command; + $break_line{$_} = 1; + $translate_line_command{$_} = 0; +} + +foreach (qw(defcv deffn + defivar defmac defmethod defop + defopt defspec deftp deftypecv + deftypefn deftypefun + deftypeivar deftypemethod + deftypeop deftypevar deftypevr + defun defvar defvr)) { + $commands{$_} = \&environment_line_command; + $translate_line_command{$_} = 1; + $break_line{$_} = 1; +} +foreach (qw(defcvx deffnx defivarx defmacx defmethodx defopx defoptx + defspecx deftpx deftypecvx deftypefnx deftypefunx deftypeivarx + deftypemethodx deftypeopx deftypevarx deftypevrx defunx + defvarx defvrx)) { + $commands{$_} = \&line_command; + $translate_line_command{$_} = 1; + $break_line{$_} = 1; +} + +foreach (qw(titlefont w i r b sansserif sc slanted strong t cite email + footnote indicateurl emph ref xref pxref inforef kbd key + acronym), +# The following commands could cause problems since their arguments +# have a semantic and a translator could decide not to translate code but +# still translate theses short words if they appear in another context. + qw(file command dfn dmn option math code samp var)) { + register_generic_command("-$_,{_}"); +} + +register_generic_command("*anchor,{_}"); +register_generic_command("*refill,"); + +$translate_line_command{'node'} = 1; +$no_wrap_environments .= " node"; +$break_line{'node'} = 1; +# @node Comments, Minimum, Conventions, Overview +$commands{'node'} = sub { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "node($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'}); + + my $translated = $ESCAPE.$command; + my $line = $args->[1]; + if (defined $line and length $line) { + my @pointers = split (/, */, $line); + my @t; + foreach (@pointers) { + push @t, $self->translate($_, $self->{ref}, $command, "wrap" => 0); + } + $translated .= " ".join(", ", @t); + } + + print "($translated,@$env)\n" + if ($debug{'commands'}); + return ($translated,@$env); +}; + +sub environment_command { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "environment_command($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'}); + my ($t,@e)=("",()); + + ($t, @e) = generic_command($self,$command,$variant,$args,$env,$no_wrap); + @e = (@$env, $command); + + print "($t,@e)\n" + if ($debug{'commands'}); + return ($t,@e); +} + +sub environment_line_command { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "environment_command_line($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'}); + my ($t,@e)=("",()); + + ($t, @e) = line_command($self,$command,$variant,$args,$env,$no_wrap); + @e = (@$env, $command); + + print "($t,@e)\n" + if ($debug{'commands'}); + return ($t,@e); +} + +## push the environment in the environment stack, and do not translate +## the command +#sub push_environment { +# my $self = shift; +# my ($command,$variant,$args,$env) = (shift,shift,shift,shift); +# print "push_environment($command,$variant,@$args,@$env)=" +# if ($debug{'environments'}); +# +# my ($t,@e) = generic_command($self,$command,$variant,$args,$env); +# +# print "($t,@e)\n" +# if ($debug{'environments'}); +# return ($t,@e); +#} +# +foreach (qw(detailmenu menu titlepage group copying + documentdescription cartouche + direntry + ifdocbook ifhtml ifinfo ifplaintext iftex ifxml + ifnotdocbook ifnothtml ifnotinfo ifnotplaintext ifnottex ifnotxml)) { + $commands{$_} = \&environment_line_command; + $translate_line_command{$_} = 0; + $break_line{$_} = 1; +} +foreach (qw(enumerate multitable ifclear ifset)) { + $commands{$_} = \&environment_line_command; + $break_line{$_} = 1; +} +foreach (qw(quotation)) { + $commands{$_} = \&environment_line_command; + $translate_line_command{$_} = 1; + $break_line{$_} = 1; +} + +$env_separators{'format'} = "(?:(?:^|\n)\\\*|END-INFO-DIR-ENTRY|START-INFO-DIR-ENTRY)"; +$env_separators{'multitable'} = "(?:\@item|\@tab)"; + +my $end_command=$commands{'end'}; +register_generic_command("*end, "); +$commands{'end'} = $end_command; +$break_line{'end'} = 1; + +register_generic_command("*macro, "); +$commands{'macro'} = \&environment_command; +$break_line{'macro'} = 1; +register_generic_command("*itemize, "); +$commands{'itemize'} = \&environment_command; +$break_line{'itemize'} = 1; +register_generic_command("*table, "); +$commands{'table'} = \&environment_command; +$break_line{'table'} = 1; + +# TODO: is_closed, use a regexp: \ does not escape the closing brace. +# TBC on LaTeX. +# In Texinfo, it appears with the "code" command. Maybe this command should +# be used as verbatim. (Expressions.texi) + +# TODO: @include @ignore + +# TBC: node Indices + +1; Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/BibTeX.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/BibTeX.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/BibTeX.pm (revision 2840) @@ -0,0 +1,152 @@ +#!/usr/bin/perl -w + +# Po4a::BibTeX.pm +# +# extract and translate translatable strings from BibTeX documents +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::BibTeX - convert BibTeX documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::BibTeX is a module to help the translation of +bibliographies in the BibTeX format into other [human] languages. + +Fields values are extracted and proposed for translation. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +NONE. + +=head1 STATUS OF THIS MODULE + +It is a very simple module, but still young. + +=cut + +package Locale::Po4a::BibTeX; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +sub initialize {} + +sub parse { + my $self = shift; + my ($line,$ref); + my $paragraph=""; + my $field=""; + my $id=""; + my $wrapped_mode = 1; + ($line,$ref)=$self->shiftline(); + while (defined($line)) { + chomp($line); +#print "tutu: '$line'\n"; + $self->{ref}="$ref"; + if ( $id eq "" + and $line =~ m/^\@.*?\s*\{\s*(.*),\s*$/) { + $id = $1; + $self->pushline( $line."\n" ); + } elsif ( $id ne "" + and $field eq "" + and $line =~ m/^((.*?)\s*=\s*)([^ "{].*?)(\s*,?\s*)$/) { + my $end=(defined $4)?$4:""; + $self->pushline( $1.$self->translate($3, + $self->{ref}, + "$2 ($id)", + "wrap" => 1).$end."\n" ); + $field = ""; + $paragraph = ""; + } elsif ( $id ne "" + and $field eq "" + and $line =~ m/^((.*?)\s*=\s*)(.*)$/) { + $field = $2; + $paragraph = $3."\n"; + $self->pushline( $1 ); + } elsif ($field ne "") { + $paragraph.="$line\n"; + } elsif ($line =~ m/^\s*(\%.*)?$/) { + $self->pushline( $line."\n" ); + } elsif ($line =~ m/^\s*\}\s*$/) { + $self->pushline( $line."\n" ); + $id=""; + } else { + print "unsupported line: '$line'\n"; + } + if ( $paragraph =~ m/^(\s*\{)(.*)(\}\s*,?\s*)$/s + or $paragraph =~ m/^(\s*")(.*)("\s*,?\s*)$/s + or $paragraph =~ m/^(\s*)([^ "{].*)(\s*,?\s*)$/s) { + $self->pushline( $1.$self->translate($2, + $self->{ref}, + "$field ($id)", + "wrap" => 1).$3); + $field=""; + $paragraph=""; + } + ($line,$ref)=$self->shiftline(); + } + if ( $paragraph =~ m/^(\s*\{)(.*)(\}\s*,?\s*)$/s + or $paragraph =~ m/^(\s*")(.*)("\s*,?\s*)$/s + or $paragraph =~ m/^(\s*)(.*)(\s*,?\s*)$/s) { + $self->pushline( $self->translate($1, + $self->{ref}, + "$field ($id)", + "wrap" => 1).$2); + $field=""; + $paragraph=""; + } +} + +sub do_paragraph { + my ($self, $paragraph, $wrap) = (shift, shift, shift); + $self->pushline( $self->translate($paragraph, + $self->{ref}, + "Plain text", + "wrap" => $wrap) ); +} + +1; + +=head1 AUTHORS + + Nicolas François + +=head1 COPYRIGHT AND LICENSE + + Copyright 2006 by Nicolas FRANÇOIS . + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/LaTeX.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/LaTeX.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/LaTeX.pm (revision 2840) @@ -0,0 +1,403 @@ +#!/usr/bin/perl -w + +# Copyright (c) 2004, 2005 by Nicolas FRANÇOIS +# +# This file is part of po4a. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with po4a; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::LaTeX - convert LaTeX documents and derivates from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::LaTeX is a module to help the translation of LaTeX documents into +other [human] languages. It can also be used as a base to build modules for +LaTeX-based documents. + +This module contains the definitions of common LaTeX commands and +environments. + +See the L manpage for the list +of recognized options. + +=head1 SEE ALSO + +L, +L, +L + +=head1 AUTHORS + + Nicolas François + +=head1 COPYRIGHT AND LICENSE + +Copyright 2004, 2005 by Nicolas FRANÇOIS . + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see COPYING file). + +=cut + +package Locale::Po4a::LaTeX; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw($VERSION @ISA @EXPORT); +$VERSION= $Locale::Po4a::TeX::VERSION; +@ISA= qw(Locale::Po4a::TeX); +@EXPORT= qw(); + +use Locale::Po4a::TeX; +use subs qw(&generic_command + &parse_definition_file + ®ister_generic_command + ®ister_generic_environment); +*parse_definition_file = \&Locale::Po4a::TeX::parse_definition_file; +*generic_command = \&Locale::Po4a::TeX::generic_command; +*register_generic_command = \&Locale::Po4a::TeX::register_generic_command; +*register_generic_environment = \&Locale::Po4a::TeX::register_generic_environment; +use vars qw($RE_ESCAPE $ESCAPE + $no_wrap_environments + %commands %environments + %separated_command %separated_environment + %command_parameters %environment_parameters + %env_separators + @exclude_include); +*RE_ESCAPE = \$Locale::Po4a::TeX::RE_ESCAPE; +*ESCAPE = \$Locale::Po4a::TeX::ESCAPE; +*no_wrap_environments = \$Locale::Po4a::TeX::no_wrap_environments; +*commands = \%Locale::Po4a::TeX::commands; +*environments = \%Locale::Po4a::TeX::environments; +*separated_command = \%Locale::Po4a::TeX::separated_command; +*separated_environment = \%Locale::Po4a::TeX::separated_environment; +*env_separators = \%Locale::Po4a::TeX::env_separators; +*exclude_include = \@Locale::Po4a::TeX::exclude_include; +*command_parameters = \%Locale::Po4a::TeX::command_parameters; +*environment_parameters = \%Locale::Po4a::TeX::environment_parameters; + + +# documentclass: +# Only read the documentclass in order to find some po4a directives. +# FIXME: The documentclass could contain translatable strings. +# Maybe it should be implemented as \include{}. +register_generic_command("*documentclass,[]{}"); +# We use register_generic_command to define the number and types of +# parameters. The function is then overwritten: +$commands{'documentclass'} = sub { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + + # Only try to parse the file. We don't want to fail or parse this file + # if it is a standard documentclass. + my $name = ($args->[0] eq '[')? $args->[3]: $args->[1]; + parse_definition_file($self, $name.".cls", 1); + + my ($t,@e) = generic_command($self,$command,$variant,$args,$env,$no_wrap); + + return ($t, @$env); +}; + +# LaTeX 2 +# I chose not to translate files, counters, lengths +register_generic_command("*addcontentsline,{}{}{_}"); +register_generic_command("address,{_}"); # lines are seperated by \\ +register_generic_command("*addtocontents,{}{_}"); +register_generic_command("*addtocounter,{}{}"); +register_generic_command("*addtolength,{}{}"); +register_generic_command("*addvspace,{}"); +register_generic_command("alph,{}"); # another language may not want this alphabet +register_generic_command("arabic,{}"); # another language may not want an arabic numbering +register_generic_command("*author,{_}"); # authors are separated by \and +register_generic_command("bibitem,[]{}"); +register_generic_command("*bibliographystyle,{}"); # BibTeX +register_generic_command("*bibliography,{}"); # BibTeX +register_generic_command("*centerline,{_}"); +register_generic_command("*caption,[]{_}"); +register_generic_command("cc,{_}"); +register_generic_command("circle,[]{}"); +register_generic_command("cite,[_]{}"); +register_generic_command("cline,{}"); +register_generic_command("closing,{_}"); +register_generic_command("dashbox,{}"); # followed by a (w,h) argument +register_generic_command("date,{_}"); +register_generic_command("*enlargethispage,{}"); +register_generic_command("ensuremath,{_}"); +register_generic_command("*fbox,{_}"); +register_generic_command("fnsymbol,{}"); +register_generic_command("*footnote,[]{_}"); +register_generic_command("*footnotemark,[]"); +register_generic_command("*footnotetext,[]{_}"); +register_generic_command("frac,{_}{_}"); +register_generic_command("*frame,{_}"); +register_generic_command("*framebox,[][]{_}"); # There is another form in picture environment +register_generic_command("*hbox,{}"); +register_generic_command("*hspace,[]{}"); +register_generic_command("*hyphenation,{_}"); # Translators may wish to add/remove words +register_generic_command("*include,{}"); +#register_generic_command("includeonly,{}"); # should not be supported for now +register_generic_command("*index,{_}"); +register_generic_command("*input,{}"); +register_generic_command("*item,[_]"); +register_generic_command("*label,{}"); +register_generic_command("lefteqn,{_}"); +register_generic_command("line,"); # The first argument is (x,y) +register_generic_command("*linebreak,[]"); +register_generic_command("linethickness,{}"); +register_generic_command("location,{_}"); +register_generic_command("makebox,[][]{_}"); # There's another form in picture environment +register_generic_command("makelabels,{}"); +register_generic_command("*markboth,[]{_}{_}"); +register_generic_command("*markright,{_}"); +register_generic_command("mathcal,{_}"); +register_generic_command("mathop,{_}"); +register_generic_command("mbox,{_}"); +register_generic_command("multicolumn,{}{}{_}"); +register_generic_command("multiput,"); # The first arguments are (x,y)(dx,dy) +register_generic_command("name,{_}"); +register_generic_command("*newcommand,{}[][]{_}"); +register_generic_command("*newcounter,{}[]"); +register_generic_command("*newenvironment,{}[]{_}{_}"); +register_generic_command("*newfont,{}{}"); +register_generic_command("*newlength,{}"); +register_generic_command("*newsavebox,{}"); +register_generic_command("*newtheorem,{}{_}"); # Two forms, the optionnal arg is not the first one +register_generic_command("nocite,{}"); +register_generic_command("nolinebreak,[]"); +register_generic_command("*nopagebreak,[]"); +register_generic_command("opening,{_}"); +register_generic_command("oval,"); # The first argument is (w,h) +register_generic_command("overbrace,{_}"); +register_generic_command("overline,{_}"); +register_generic_command("*pagebreak,[]"); +register_generic_command("*pagenumbering,{_}"); +register_generic_command("pageref,{}"); +register_generic_command("*pagestyle,{}"); +register_generic_command("*parbox,[][][]{}{_}"); +register_generic_command("providecommand,{}[][]{_}"); +register_generic_command("put,"); # The first argument is (x,y) +register_generic_command("raisebox,{}[][]{_}"); +register_generic_command("ref,{}"); +register_generic_command("*refstepcounter,{}"); +register_generic_command("*renewcommand,{}[][]{_}"); +register_generic_command("*renewenvironment,{}[]{_}{_}"); +register_generic_command("roman,{}"); # another language may not want a roman numbering +register_generic_command("rule,[]{}{}"); +register_generic_command("savebox,{}"); # Optional arguments in 2nd & 3rd position +register_generic_command("sbox,{}{_}"); +register_generic_command("*setcounter,{}{}"); +register_generic_command("*setlength,{}{}"); +register_generic_command("*settodepth,{}{_}"); +register_generic_command("*settoheight,{}{_}"); +register_generic_command("*settowidth,{}{_}"); +register_generic_command("shortstack,[]{_}"); +register_generic_command("signature,{_}"); +register_generic_command("sqrt,[_]{_}"); +register_generic_command("stackrel,{_}{_}"); +register_generic_command("stepcounter,{}"); +register_generic_command("*subfigure,[_]{_}"); +register_generic_command("symbol,{_}"); +register_generic_command("telephone,{_}"); +register_generic_command("thanks,{_}"); +register_generic_command("*thispagestyle,{}"); +register_generic_command("*title,{_}"); +register_generic_command("typeout,{_}"); +register_generic_command("typein,[]{_}"); +register_generic_command("twocolumn,[_]"); +register_generic_command("underbrace,{_}"); +register_generic_command("underline,{_}"); +register_generic_command("*usebox,{}"); +register_generic_command("usecounter,{}"); +register_generic_command("*usepackage,[]{}"); +register_generic_command("value,{}"); +register_generic_command("vector,"); # The first argument is (x,y) +register_generic_command("vphantom,{_}"); +register_generic_command("*vspace,[]{}"); +register_generic_command("*vbox,{}"); +register_generic_command("*vcenter,{}"); + +register_generic_command("*part,[_]{_}"); +register_generic_command("*chapter,[_]{_}"); +register_generic_command("*section,[_]{_}"); +register_generic_command("*subsection,[_]{_}"); +register_generic_command("*subsubsection,[_]{_}"); +register_generic_command("*paragraph,[_]{_}"); +register_generic_command("*subparagraph,[_]{_}"); + +register_generic_command("textrm,{_}"); +register_generic_command("textit,{_}"); +register_generic_command("emph,{_}"); +register_generic_command("textmd,{_}"); +register_generic_command("textbf,{_}"); +register_generic_command("textup,{_}"); +register_generic_command("textsl,{_}"); +register_generic_command("textsf,{_}"); +register_generic_command("textsc,{_}"); +register_generic_command("texttt,{_}"); +register_generic_command("textnormal,{_}"); +register_generic_command("mathrm,{_}"); +register_generic_command("mathsf,{_}"); +register_generic_command("mathtt,{_}"); +register_generic_command("mathit,{_}"); +register_generic_command("mathnormal,{_}"); +register_generic_command("mathversion,{}"); + +register_generic_command("*contentspage,"); +register_generic_command("*tablelistpage,"); +register_generic_command("*figurepage,"); + +register_generic_command("*PassOptionsToPackage,{}{}"); + +register_generic_command("*ifthenelse,{}{_}{_}"); + +# graphics +register_generic_command("*includegraphics,[]{}"); +register_generic_command("*graphicspath,{}"); +register_generic_command("*resizebox,{}{}{_}"); +register_generic_command("*scalebox,{}{_}"); +register_generic_command("*rotatebox,{}{_}"); + +# url +register_generic_command("UrlFont,{}"); +register_generic_command("*urlstyle,{}"); + +# hyperref +register_generic_command("href,{}{_}"); # 1:URL +register_generic_command("url,{}"); # URL +register_generic_command("nolinkurl,{}"); # URL +register_generic_command("hyperbaseurl,{}"); # URL +register_generic_command("hyperimage,{}"); # URL +register_generic_command("hyperdef,{}{}{_}"); # 1:category, 2:name +register_generic_command("hyperref,{}{}{}{_}"); # 1:URL, 2:category, 3:name +register_generic_command("hyperlink,{}{_}"); # 1:name +register_generic_command("*hypersetup,{_}"); +register_generic_command("hypertarget,{}{_}"); # 1:name +register_generic_command("autoref,{}"); # 1:label + +register_generic_command("*selectlanguage,{}"); + +# color +register_generic_command("*definecolor,{}{}{}"); +register_generic_command("*textcolor,{}{_}"); +register_generic_command("*colorbox,{}{_}"); +register_generic_command("*fcolorbox,{}{}{_}"); +register_generic_command("*pagecolor,{_}"); +register_generic_command("*color,{}"); + +# equations/theorems +register_generic_command("*qedhere,"); +register_generic_command("*qedsymbol,"); +register_generic_command("*theoremstyle,{}"); +register_generic_command("*proclaim,{_}"); +register_generic_command("*endproclaim,"); +register_generic_command("*shoveleft,{_}"); +register_generic_command("*shoveright,{_}"); + +# commands without arguments. This is better than untranslated or +# translate_joined because the number of arguments will be checked. +foreach (qw(a *appendix *backmatter backslash *baselineskip *baselinestretch bf + *bigskip boldmath cal cdots *centering *cleardoublepage *clearpage + ddots dotfill em flushbottom *footnotesize frenchspacing + *frontmatter *glossary *hfill *hline hrulefill huge Huge indent it + kill large Large LARGE ldots left linewidth listoffigures + listoftables *mainmatter *makeatletter *makeglossary *makeindex + *maketitle *medskip *newline *newpage noindent nonumber *normalsize + not *null *onecolumn *par parindent *parskip *printindex protect ps + pushtabs *qquad *quad raggedbottom raggedleft raggedright right rm + sc scriptsize sf sl small *smallskip *startbreaks *stopbreaks + *tableofcontents textwidth textheight tiny today tt unitlength + vdots verb *vfill *vline *fussy *sloppy + + aleph hbar imath jmath ell wp Re Im prime nabla surd angle forall + exists partial infty triangle Box Diamond flat natural sharp + clubsuit diamondsuit heartsuit spadesuit dag ddag S P copyright + pounds Delta ASCII + + rmfamily itshape mdseries bfseries upshape slshape sffamily scshape + ttfamily *normalfont width height depth totalheight + + *fboxsep *fboxrule + *itemi *itemii *itemiii *itemiv + *theitemi *theitemii *theitemiii *theitemiv)) { + register_generic_command("$_,"); +} + + + +# standard environments. +# FIXME: All these definitions should be re-checked +foreach (qw(abstract align align* cases center description displaymath document enumerate + eqnarray eqnarray* equation equation* flushleft flushright footnotesize itemize + letter lrbox multline multline* proof quotation quote + sloppypar tabbing theorem titlepage + trivlist verbatim verbatim* verse wrapfigure)) { + register_generic_environment("$_,"); +} +register_generic_environment("tabular,[]{}"); +register_generic_environment("tabular*,{}{}"); +register_generic_environment("tabularx,{}{}"); +register_generic_environment("multicols,{}"); +register_generic_environment("list,{_}{}"); +register_generic_environment("array,[]{}"); +register_generic_environment("figure,[]"); +register_generic_environment("minipage,[]{}"); +register_generic_environment("picture,{}{}"); +register_generic_environment("table,[]"); +register_generic_environment("thebibliography,{_}"); + + +# Commands and environments with separators. + +# & is the cell separator, \\ is the line separator +# '\' is escaped twice +$env_separators{'array'} = + $env_separators{'tabular'} = + $env_separators{'tabularx'} = "(?:&|\\\\\\\\|\\\\hline)"; + +$env_separators{'trivlist'} = + $env_separators{'list'} = + $env_separators{'description'} = + $env_separators{'enumerate'} = + $env_separators{'itemize'} = "\\\\item"; + +$env_separators{'thebibliography'} = "\\\\bibitem"; + +$env_separators{'displaymath'} = + $env_separators{'eqnarray'} = + $env_separators{'eqnarray*'} = + $env_separators{'flushleft'} = + $env_separators{'flushright'} = + $env_separators{'center'} = + $env_separators{'author{#1}'} = + $env_separators{'title{#1}'} = "\\\\\\\\"; + +# tabbing + +1; Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Po.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Po.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Po.pm (revision 2840) @@ -0,0 +1,1664 @@ +# Locale::Po4a::Po -- manipulation of PO files +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING). + +############################################################################ +# Modules and declarations +############################################################################ + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Po - PO file manipulation module + +=head1 SYNOPSIS + + use Locale::Po4a::Po; + my $pofile=Locale::Po4a::Po->new(); + + # Read PO file + $pofile->read('file.po'); + + # Add an entry + $pofile->push('msgid' => 'Hello', 'msgstr' => 'bonjour', + 'flags' => "wrap", 'reference'=>'file.c:46'); + + # Extract a translation + $pofile->gettext("Hello"); # returns 'bonjour' + + # Write back to a file + $pofile->write('otherfile.po'); + +=head1 DESCRIPTION + +Locale::Po4a::Po is a module that allows you to manipulate message +catalogs. You can load and write from/to a file (which extension is often +I), you can build new entries on the fly or request for the translation +of a string. + +For a more complete description of message catalogs in the PO format and +their use, please refer to the documentation of the gettext program. + +This module is part of the po4a project, which objective is to use PO files +(designed at origin to ease the translation of program messages) to +translate everything, including documentation (man page, info manual), +package description, debconf templates, and everything which may benefit +from this. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +=over 4 + +=item B I[,B|B] + +Specify the reference format. Argument I can be one of B to not +produce any reference, B to not specify the line number (more +accurately all line numbers are replaced by 1), B to replace line +number by an increasing counter, and B to include complete +references. + +Argument can be followed by a comma and either B or B keyword. +References are written by default on a single line. The B option wraps +references on several lines, to mimic B tools (B and +B). This option will become the default in a future release, because +it is more sensible. The B option is available so that users who want +to keep the old behavior can do so. + +=item B<--msgid-bugs-address> I + +Set the report address for msgid bugs. By default, the created POT files +have no Report-Msgid-Bugs-To fields. + +=item B<--copyright-holder> I + +Set the copyright holder in the POT header. The default value is +"Free Software Foundation, Inc." + +=item B<--package-name> I + +Set the package name for the POT header. The default is "PACKAGE". + +=item B<--package-version> I + +Set the package version for the POT header. The default is "VERSION". + +=back + +=cut + +use IO::File; + + +require Exporter; + +package Locale::Po4a::Po; +use DynaLoader; + +use Locale::Po4a::Common qw(wrap_msg wrap_mod wrap_ref_mod dgettext); + +use subs qw(makespace); +use vars qw(@ISA @EXPORT_OK); +@ISA = qw(Exporter DynaLoader); +@EXPORT = qw(%debug); +@EXPORT_OK = qw(&move_po_if_needed); + +use Locale::Po4a::TransTractor; +# Try to use a C extension if present. +eval("bootstrap Locale::Po4a::Po $Locale::Po4a::TransTractor::VERSION"); + +use 5.006; +use strict; +use warnings; + +use Carp qw(croak); +use File::Basename; +use File::Path; # mkdir before write +use File::Copy; # move +use POSIX qw(strftime floor); +use Time::Local; + +use Encode; + +my @known_flags=qw(wrap no-wrap c-format fuzzy); + +our %debug=('canonize' => 0, + 'quote' => 0, + 'escape' => 0, + 'encoding' => 0, + 'filter' => 0); + +=head1 Functions about whole message catalogs + +=over 4 + +=item new() + +Creates a new message catalog. If an argument is provided, it's the name of +a PO file we should load. + +=cut + +sub new { + my ($this, $options) = (shift, shift); + my $class = ref($this) || $this; + my $self = {}; + bless $self, $class; + $self->initialize($options); + + my $filename = shift; + $self->read($filename) if defined($filename) && length($filename); + return $self; +} + +# Return the numerical timezone (e.g. +0200) +# Neither the %z nor the %s formats of strftime are portable: +# '%s' is not supported on Solaris and '%z' indicates +# "2006-10-25 19:36E. Europe Standard Time" on MS Windows. +sub timezone { + my @g = gmtime(); + my @l = localtime(); + + my $diff; + $diff = floor(timelocal(@l)/60 +0.5); + $diff -= floor(timelocal(@g)/60 +0.5); + + my $h = floor($diff / 60) + $l[8]; # $l[8] indicates if we are currently + # in a daylight saving time zone + my $m = $diff%60; + + return sprintf "%+03d%02d\n", $h, $m; +} + +sub initialize { + my ($self, $options) = (shift, shift); + my $date = strftime("%Y-%m-%d %H:%M", localtime).timezone(); + chomp $date; +# $options = ref($options) || $options; + + $self->{options}{'porefs'}= 'full,nowrap'; + $self->{options}{'msgid-bugs-address'}= undef; + $self->{options}{'copyright-holder'}= "Free Software Foundation, Inc."; + $self->{options}{'package-name'}= "PACKAGE"; + $self->{options}{'package-version'}= "VERSION"; + foreach my $opt (keys %$options) { + if ($options->{$opt}) { + die wrap_mod("po4a::po", + dgettext ("po4a", "Unknown option: %s"), $opt) + unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options->{$opt}; + } + } + $self->{options}{'porefs'} =~ /^(full|counter|noline|none)(,(no)?wrap)?$/ || + die wrap_mod("po4a::po", + dgettext ("po4a", + "Invalid value for option 'porefs' ('%s' is ". + "not one of 'full', 'counter', 'noline' or 'none')"), + $self->{options}{'porefs'}); + if ($self->{options}{'porefs'} =~ m/^counter/) { + $self->{counter} = {}; + } + + $self->{po}=(); + $self->{count}=0; # number of msgids in the PO + # count_doc: number of strings in the document + # (duplicate strings counted multiple times) + $self->{count_doc}=0; + $self->{header_comment}= + " SOME DESCRIPTIVE TITLE\n" + ." Copyright (C) YEAR ". + $self->{options}{'copyright-holder'}."\n" + ." This file is distributed under the same license ". + "as the ".$self->{options}{'package-name'}." package.\n" + ." FIRST AUTHOR , YEAR.\n" + ."\n" + .", fuzzy"; +# $self->header_tag="fuzzy"; + $self->{header}=escape_text("Project-Id-Version: ". + $self->{options}{'package-name'}." ". + $self->{options}{'package-version'}."\n". + ((defined $self->{options}{'msgid-bugs-address'})? + "Report-Msgid-Bugs-To: ".$self->{options}{'msgid-bugs-address'}."\n": + ""). + "POT-Creation-Date: $date\n". + "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n". + "Last-Translator: FULL NAME \n". + "Language-Team: LANGUAGE \n". + "Language: \n". + "MIME-Version: 1.0\n". + "Content-Type: text/plain; charset=CHARSET\n". + "Content-Transfer-Encoding: 8bit\n"); + + $self->{encoder}=find_encoding("ascii"); + $self->{footer}=[]; + + # To make stats about gettext hits + $self->stats_clear(); +} + +=item read($) + +Reads a PO file (which name is given as argument). Previously existing +entries in self are not removed, the new ones are added to the end of the +catalog. + +=cut + +sub read { + my $self=shift; + my $filename=shift + or croak wrap_mod("po4a::po", + dgettext("po4a", + "Please provide a non-null filename")); + + my $lang = basename($filename); + $lang =~ s/\.po$//; + $self->{lang} = $lang; + + my $fh; + if ($filename eq '-') { + $fh=*STDIN; + } else { + open $fh,"<$filename" + or croak wrap_mod("po4a::po", + dgettext("po4a", "Can't read from %s: %s"), + $filename, $!); + } + + ## Read paragraphs line-by-line + my $pofile=""; + my $textline; + while (defined ($textline = <$fh>)) { + $pofile .= $textline; + } +# close INPUT +# or croak (sprintf(dgettext("po4a", +# "Can't close %s after reading: %s"), +# $filename,$!)."\n"); + + my $linenum=0; + + foreach my $msg (split (/\n\n/,$pofile)) { + my ($msgid,$msgstr,$comment,$previous,$automatic,$reference,$flags,$buffer); + my ($msgid_plural, $msgstr_plural); + if ($msg =~ m/^#~/m) { + push(@{$self->{footer}}, $msg); + next; + } + foreach my $line (split (/\n/,$msg)) { + $linenum++; + if ($line =~ /^#\. ?(.*)$/) { # Automatic comment + $automatic .= (defined($automatic) ? "\n" : "").$1; + + } elsif ($line =~ /^#: ?(.*)$/) { # reference + $reference .= (defined($reference) ? "\n" : "").$1; + + } elsif ($line =~ /^#, ?(.*)$/) { # flags + $flags .= (defined($flags) ? "\n" : "").$1; + + } elsif ($line =~ /^#\| ?(.*)$/) { # previous translation + $previous .= (defined($previous) ? "\n" : "").($1||""); + + } elsif ($line =~ /^#(.*)$/) { # Translator comments + $comment .= (defined($comment) ? "\n" : "").($1||""); + + } elsif ($line =~ /^msgid (".*")$/) { # begin of msgid + $buffer = $1; + + } elsif ($line =~ /^msgid_plural (".*")$/) { + # begin of msgid_plural, end of msgid + + $msgid = $buffer; + $buffer = $1; + + } elsif ($line =~ /^msgstr (".*")$/) { + # begin of msgstr, end of msgid + + $msgid = $buffer; + $buffer = "$1"; + + } elsif ($line =~ /^msgstr\[([0-9]+)\] (".*")$/) { + # begin of msgstr[x], end of msgid_plural or msgstr[x-1] + + # Note: po4a cannot uses plural forms + # (no integer to use the plural form) + # * drop the msgstr[x] where x >= 2 + # * use msgstr[0] as the translation of msgid + # * use msgstr[1] as the translation of msgid_plural + + if ($1 eq "0") { + $msgid_plural = $buffer; + $buffer = "$2"; + } elsif ($1 eq "1") { + $msgstr = $buffer; + $buffer = "$2"; + } elsif ($1 eq "2") { + $msgstr_plural = $buffer; + warn wrap_ref_mod("$filename:$linenum", + "po4a::po", + dgettext("po4a", "Messages with more than 2 plural forms are not supported.")); + } + } elsif ($line =~ /^(".*")$/) { + # continuation of a line + $buffer .= "\n$1"; + + } else { + warn wrap_ref_mod("$filename:$linenum", + "po4a::po", + dgettext("po4a", "Strange line: -->%s<--"), + $line); + } + } + $linenum++; + if (defined $msgid_plural) { + $msgstr_plural=$buffer; + + $msgid = unquote_text($msgid) if (defined($msgid)); + $msgstr = unquote_text($msgstr) if (defined($msgstr)); + + $self->push_raw ('msgid' => $msgid, + 'msgstr' => $msgstr, + 'reference' => $reference, + 'flags' => $flags, + 'comment' => $comment, + 'previous' => $previous, + 'automatic' => $automatic, + 'plural' => 0); + + $msgid_plural = unquote_text($msgid_plural) + if (defined($msgid_plural)); + $msgstr_plural = unquote_text($msgstr_plural) + if (defined($msgstr_plural)); + + $self->push_raw ('msgid' => $msgid_plural, + 'msgstr' => $msgstr_plural, + 'reference' => $reference, + 'flags' => $flags, + 'comment' => $comment, + 'previous' => $previous, + 'automatic' => $automatic, + 'plural' => 1); + } else { + $msgstr=$buffer; + + $msgid = unquote_text($msgid) if (defined($msgid)); + $msgstr = unquote_text($msgstr) if (defined($msgstr)); + + $self->push_raw ('msgid' => $msgid, + 'msgstr' => $msgstr, + 'reference' => $reference, + 'flags' => $flags, + 'comment' => $comment, + 'previous' => $previous, + 'automatic' => $automatic); + } + } +} + +=item write($) + +Writes the current catalog to the given file. + +=cut + +sub write{ + my $self=shift; + my $filename=shift + or croak dgettext("po4a","Can't write to a file without filename")."\n"; + + my $fh; + if ($filename eq '-') { + $fh=\*STDOUT; + } else { + # make sure the directory in which we should write the localized + # file exists + my $dir = $filename; + if ($dir =~ m|/|) { + $dir =~ s|/[^/]*$||; + + File::Path::mkpath($dir, 0, 0755) # Croaks on error + if (length ($dir) && ! -e $dir); + } + open $fh,">$filename" + or croak wrap_mod("po4a::po", + dgettext("po4a", "Can't write to %s: %s"), + $filename, $!); + } + + print $fh "".format_comment($self->{header_comment},"") + if defined($self->{header_comment}) && length($self->{header_comment}); + + print $fh "msgid \"\"\n"; + print $fh "msgstr ".quote_text($self->{header})."\n\n"; + + + my $buf_msgstr_plural; # Used to keep the first msgstr of plural forms + my $first=1; + foreach my $msgid ( sort { ($self->{po}{"$a"}{'pos'}) <=> + ($self->{po}{"$b"}{'pos'}) + } keys %{$self->{po}}) { + my $output=""; + + if ($first) { + $first=0; + } else { + $output .= "\n"; + } + + $output .= format_comment($self->{po}{$msgid}{'comment'},"") + if defined($self->{po}{$msgid}{'comment'}) + && length ($self->{po}{$msgid}{'comment'}); + if ( defined($self->{po}{$msgid}{'automatic'}) + && length ($self->{po}{$msgid}{'automatic'})) { + foreach my $comment (split(/\\n/,$self->{po}{$msgid}{'automatic'})) + { + $output .= format_comment($comment, ". ") + } + } + $output .= format_comment($self->{po}{$msgid}{'type'},". type: ") + if defined($self->{po}{$msgid}{'type'}) + && length ($self->{po}{$msgid}{'type'}); + if ( defined($self->{po}{$msgid}{'reference'}) + && length ($self->{po}{$msgid}{'reference'})) { + my $output_ref = $self->{po}{$msgid}{'reference'}; + if ($self->{options}{'porefs'} =~ m/,wrap$/) { + $output_ref = wrap($output_ref); + $output_ref =~ s/\s+$//mg; + } + $output .= format_comment($output_ref,": "); + } + $output .= "#, ". join(", ", sort split(/\s+/,$self->{po}{$msgid}{'flags'}))."\n" + if defined($self->{po}{$msgid}{'flags'}) + && length ($self->{po}{$msgid}{'flags'}); + $output .= format_comment($self->{po}{$msgid}{'previous'},"| ") + if defined($self->{po}{$msgid}{'previous'}) + && length ($self->{po}{$msgid}{'previous'}); + + if (exists $self->{po}{$msgid}{'plural'}) { + if ($self->{po}{$msgid}{'plural'} == 0) { + if ($self->get_charset =~ /^utf-8$/i) { + my $msgstr = Encode::decode_utf8($self->{po}{$msgid}{'msgstr'}); + $msgid = Encode::decode_utf8($msgid); + $output .= Encode::encode_utf8("msgid ".quote_text($msgid)."\n"); + $buf_msgstr_plural = Encode::encode_utf8("msgstr[0] ".quote_text($msgstr)."\n"); + } else { + $output = "msgid ".quote_text($msgid)."\n"; + $buf_msgstr_plural = "msgstr[0] ".quote_text($self->{po}{$msgid}{'msgstr'})."\n"; + } + } elsif ($self->{po}{$msgid}{'plural'} == 1) { +# TODO: there may be only one plural form + if ($self->get_charset =~ /^utf-8$/i) { + my $msgstr = Encode::decode_utf8($self->{po}{$msgid}{'msgstr'}); + $msgid = Encode::decode_utf8($msgid); + $output = Encode::encode_utf8("msgid_plural ".quote_text($msgid)."\n"); + $output .= $buf_msgstr_plural; + $output .= Encode::encode_utf8("msgstr[1] ".quote_text($msgstr)."\n"); + $buf_msgstr_plural = ""; + } else { + $output = "msgid_plural ".quote_text($msgid)."\n"; + $output .= $buf_msgstr_plural; + $output .= "msgstr[1] ".quote_text($self->{po}{$msgid}{'msgstr'})."\n"; + } + } else { + die wrap_msg(dgettext("po4a","Can't write PO files with more than two plural forms.")); + } + } else { + if ($self->get_charset =~ /^utf-8$/i) { + my $msgstr = Encode::decode_utf8($self->{po}{$msgid}{'msgstr'}); + $msgid = Encode::decode_utf8($msgid); + $output .= Encode::encode_utf8("msgid ".quote_text($msgid)."\n"); + $output .= Encode::encode_utf8("msgstr ".quote_text($msgstr)."\n"); + } else { + $output .= "msgid ".quote_text($msgid)."\n"; + $output .= "msgstr ".quote_text($self->{po}{$msgid}{'msgstr'})."\n"; + } + } + + print $fh $output; + } + print $fh join("\n\n", @{$self->{footer}}) if scalar @{$self->{footer}}; + +# print STDERR "$fh"; +# if ($filename ne '-') { +# close $fh +# or croak (sprintf(dgettext("po4a", +# "Can't close %s after writing: %s\n"), +# $filename,$!)); +# } +} + +=item write_if_needed($$) + +Like write, but if the PO or POT file already exists, the object will be +written in a temporary file which will be compared with the existing file +to check if the update is needed (this avoids to change a POT just to +update a line reference or the POT-Creation-Date field). + +=cut + +sub move_po_if_needed { + my ($new_po, $old_po, $backup) = (shift, shift, shift); + my $diff; + + if (-e $old_po) { + my $diff_ignore = "-I'^#:' " + ."-I'^\"POT-Creation-Date:' " + ."-I'^\"PO-Revision-Date:'"; + $diff = qx(diff -q $diff_ignore $old_po $new_po); + if ( $diff eq "" ) { + unlink $new_po + or die wrap_msg(dgettext("po4a","Can't unlink %s: %s."), + $new_po, $!); + # touch the old PO + my ($atime, $mtime) = (time,time); + utime $atime, $mtime, $old_po; + } else { + move $new_po, $old_po + or die wrap_msg(dgettext("po4a","Can't move %s to %s: %s."), + $new_po, $old_po, $!); + } + } else { + move $new_po, $old_po + or die wrap_msg(dgettext("po4a","Can't move %s to %s: %s."), + $new_po, $old_po, $!); + } +} + +sub write_if_needed { + my $self=shift; + my $filename=shift + or croak dgettext("po4a","Can't write to a file without filename")."\n"; + + if (-e $filename) { + my ($tmp_filename); + (undef,$tmp_filename)=File::Temp->tempfile($filename."XXXX", + DIR => "/tmp", + OPEN => 0, + UNLINK => 0); + $self->write($tmp_filename); + move_po_if_needed($tmp_filename, $filename); + } else { + $self->write($filename); + } +} + +=item gettextize($$) + +This function produces one translated message catalog from two catalogs, an +original and a translation. This process is described in L, +section I. + +=cut + +sub gettextize { + my $this = shift; + my $class = ref($this) || $this; + my ($poorig,$potrans)=(shift,shift); + + my $pores=Locale::Po4a::Po->new(); + + my $please_fail = 0; + my $toobad = dgettext("po4a", + "\nThe gettextization failed (once again). Don't give up, ". + "gettextizing is a subtle art, but this is only needed once ". + "to convert a project to the gorgeous luxus offered by po4a ". + "to translators.". + "\nPlease refer to the po4a(7) documentation, the section ". + "\"HOWTO convert a pre-existing translation to po4a?\" ". + "contains several hints to help you in your task"); + + # Don't fail right now when the entry count does not match. Instead, give + # it a try so that the user can see where we fail (which is probably where + # the problem is). + if ($poorig->count_entries_doc() > $potrans->count_entries_doc()) { + warn wrap_mod("po4a gettextize", dgettext("po4a", + "Original has more strings than the translation (%d>%d). ". + "Please fix it by editing the translated version to add ". + "some dummy entry."), + $poorig->count_entries_doc(), + $potrans->count_entries_doc()); + $please_fail = 1; + } elsif ($poorig->count_entries_doc() < $potrans->count_entries_doc()) { + warn wrap_mod("po4a gettextize", dgettext("po4a", + "Original has less strings than the translation (%d<%d). ". + "Please fix it by removing the extra entry from the ". + "translated file. You may need an addendum (cf po4a(7)) ". + "to reput the chunk in place after gettextization. A ". + "possible cause is that a text duplicated in the original ". + "is not translated the same way each time. Remove one of ". + "the translations, and you're fine."), + $poorig->count_entries_doc(), + $potrans->count_entries_doc()); + $please_fail = 1; + } + + if ( $poorig->get_charset =~ /^utf-8$/i ) { + $potrans->to_utf8; + $pores->set_charset("UTF-8"); + } else { + if ($potrans->get_charset eq "CHARSET") { + $pores->set_charset("ascii"); + } else { + $pores->set_charset($potrans->get_charset); + } + } + print "Po character sets:\n". + " original=".$poorig->get_charset."\n". + " translated=".$potrans->get_charset."\n". + " result=".$pores->get_charset."\n" + if $debug{'encoding'}; + + for (my ($o,$t)=(0,0) ; + $o<$poorig->count_entries_doc() && $t<$potrans->count_entries_doc(); + $o++,$t++) { + # + # Extract some informations + + my ($orig,$trans)=($poorig->msgid_doc($o),$potrans->msgid_doc($t)); +# print STDERR "Matches [[$orig]]<<$trans>>\n"; + + my ($reforig,$reftrans)=($poorig->{po}{$orig}{'reference'}, + $potrans->{po}{$trans}{'reference'}); + my ($typeorig,$typetrans)=($poorig->{po}{$orig}{'type'}, + $potrans->{po}{$trans}{'type'}); + + # + # Make sure the type of both string exist + # + die wrap_mod("po4a gettextize", + "Internal error: type of original string number %s ". + "isn't provided", $o) + if ($typeorig eq ''); + + die wrap_mod("po4a gettextize", + "Internal error: type of translated string number %s ". + "isn't provided", $o) + if ($typetrans eq ''); + + # + # Make sure both type are the same + # + if ($typeorig ne $typetrans){ + $pores->write("gettextization.failed.po"); + eval { + # Recode $trans into current charset, if possible + require I18N::Langinfo; + I18N::Langinfo->import(qw(langinfo CODESET)); + my $codeset = langinfo(CODESET()); + Encode::from_to($trans, $potrans->get_charset, $codeset); + }; + die wrap_msg(dgettext("po4a", + "po4a gettextization: Structure disparity between ". + "original and translated files:\n". + "msgid (at %s) is of type '%s' while\n". + "msgstr (at %s) is of type '%s'.\n". + "Original text: %s\n". + "Translated text: %s\n". + "(result so far dumped to gettextization.failed.po)"). + "%s", + $reforig, $typeorig, + $reftrans, $typetrans, + $orig, + $trans, + $toobad); + } + + # + # Push the entry + # + my $flags; + if (defined $poorig->{po}{$orig}{'flags'}) { + $flags = $poorig->{po}{$orig}{'flags'}." fuzzy"; + } else { + $flags = "fuzzy"; + } + $pores->push_raw('msgid' => $orig, + 'msgstr' => $trans, + 'flags' => $flags, + 'type' => $typeorig, + 'reference' => $reforig, + 'conflict' => 1, + 'transref' => $potrans->{po}{$trans}{'reference'}) + unless (defined($pores->{po}{$orig}) + and ($pores->{po}{$orig}{'msgstr'} eq $trans)) + # FIXME: maybe we should be smarter about what reference should be + # sent to push_raw. + } + + # make sure we return a useful error message when entry count differ + die "$toobad\n" if $please_fail; + + return $pores; +} + +=item filter($) + +This function extracts a catalog from an existing one. Only the entries having +a reference in the given file will be placed in the resulting catalog. + +This function parses its argument, converts it to a Perl function definition, +evals this definition and filters the fields for which this function returns +true. + +I love Perl sometimes ;) + +=cut + +sub filter { + my $self=shift; + our $filter=shift; + + my $res; + $res = Locale::Po4a::Po->new(); + + # Parse the filter + our $code="sub apply { return "; + our $pos=0; + our $length = length $filter; + + # explode chars to parts. How to subscript a string in Perl? + our @filter = split(//,$filter); + + sub gloups { + my $fmt=shift; + my $space = ""; + for (1..$pos){ + $space .= ' '; + } + die wrap_msg("$fmt\n$filter\n$space^ HERE"); + } + sub showmethecode { + return unless $debug{'filter'}; + my $fmt=shift; + my $space=""; + for (1..$pos){ + $space .= ' '; + } + print STDERR "$filter\n$space^ $fmt\n";#"$code\n"; + } + + # I dream of a lex in perl :-/ + sub parse_expression { + showmethecode("Begin expression") + if $debug{'filter'}; + + gloups("Begin of expression expected, got '%s'",$filter[$pos]) + unless ($filter[$pos] eq '('); + $pos ++; # pass the '(' + if ($filter[$pos] eq '&') { + # AND + $pos++; + showmethecode("Begin of AND") + if $debug{'filter'}; + $code .= "("; + while (1) { + gloups ("Unfinished AND statement.") + if ($pos == $length); + parse_expression(); + if ($filter[$pos] eq '(') { + $code .= " && "; + } elsif ($filter[$pos] eq ')') { + last; # do not eat that char + } else { + gloups("End of AND or begin of sub-expression expected, got '%s'", $filter[$pos]); + } + } + $code .= ")"; + } elsif ($filter[$pos] eq '|') { + # OR + $pos++; + $code .= "("; + while (1) { + gloups("Unfinished OR statement.") + if ($pos == $length); + parse_expression(); + if ($filter[$pos] eq '(') { + $code .= " || "; + } elsif ($filter[$pos] eq ')') { + last; # do not eat that char + } else { + gloups("End of OR or begin of sub-expression expected, got '%s'",$filter[$pos]); + } + } + $code .= ")"; + } elsif ($filter[$pos] eq '!') { + # NOT + $pos++; + $code .= "(!"; + gloups("Missing sub-expression in NOT statement.") + if ($pos == $length); + parse_expression(); + $code .= ")"; + } else { + # must be an equal. Let's get field and argument + my ($field,$arg,$done); + $field = substr($filter,$pos); + gloups("EQ statement contains no '=' or invalid field name") + unless ($field =~ /([a-z]*)=/i); + $field = lc($1); + $pos += (length $field) + 1; + + # check that we've got a valid field name, + # and the number it referes to + # DO NOT CHANGE THE ORDER + my @names=qw(msgid msgstr reference flags comment previous automatic); + my $fieldpos; + for ($fieldpos = 0; + $fieldpos < scalar @names && $field ne $names[$fieldpos]; + $fieldpos++) {} + gloups("Invalid field name: %s",$field) + if $fieldpos == scalar @names; # not found + + # Now, get the argument value. It has to be between quotes, + # which can be escaped + # We point right on the first char of the argument + # (first quote already eaten) + my $escaped = 0; + my $quoted = 0; + if ($filter[$pos] eq '"') { + $pos++; + $quoted = 1; + } + showmethecode(($quoted?"Quoted":"Unquoted")." argument of field '$field'") + if $debug{'filter'}; + + while (!$done) { + gloups("Unfinished EQ argument.") + if ($pos == $length); + + if ($quoted) { + if ($filter[$pos] eq '\\') { + if ($escaped) { + $arg .= '\\'; + $escaped = 0; + } else { + $escaped = 1; + } + } elsif ($escaped) { + if ($filter[$pos] eq '"') { + $arg .= '"'; + $escaped = 0; + } else { + gloups("Invalid escape sequence in argument: '\\%s'",$filter[$pos]); + } + } else { + if ($filter[$pos] eq '"') { + $done = 1; + } else { + $arg .= $filter[$pos]; + } + } + } else { + if ($filter[$pos] eq ')') { + # counter the next ++ since we don't want to eat + # this char + $pos--; + $done = 1; + } else { + $arg .= $filter[$pos]; + } + } + $pos++; + } + # and now, add the code to check this equality + $code .= "(\$_[$fieldpos] =~ m{$arg})"; + + } + showmethecode("End of expression") + if $debug{'filter'}; + gloups("Unfinished statement.") + if ($pos == $length); + gloups("End of expression expected, got '%s'",$filter[$pos]) + unless ($filter[$pos] eq ')'); + $pos++; + } + # And now, launch the beast, finish the function and use eval + # to construct this function. + # Ok, the lack of lexer is a fair price for the eval ;) + parse_expression(); + gloups("Garbage at the end of the expression") + if ($pos != $length); + $code .= "; }"; + print STDERR "CODE = $code\n" + if $debug{'filter'}; + eval $code; + die wrap_mod("po4a::po", dgettext("po4a", "Eval failure: %s"), $@) + if $@; + + for (my $cpt=(0) ; + $cpt<$self->count_entries(); + $cpt++) { + + my ($msgid,$ref,$msgstr,$flags,$type,$comment,$previous,$automatic); + + $msgid = $self->msgid($cpt); + $ref=$self->{po}{$msgid}{'reference'}; + + $msgstr= $self->{po}{$msgid}{'msgstr'}; + $flags = $self->{po}{$msgid}{'flags'}; + $type = $self->{po}{$msgid}{'type'}; + $comment = $self->{po}{$msgid}{'comment'}; + $previous = $self->{po}{$msgid}{'previous'}; + $automatic = $self->{po}{$msgid}{'automatic'}; + + # DO NOT CHANGE THE ORDER + $res->push_raw('msgid' => $msgid, + 'msgstr' => $msgstr, + 'flags' => $flags, + 'type' => $type, + 'reference' => $ref, + 'comment' => $comment, + 'previous' => $previous, + 'automatic' => $automatic) + if (apply($msgid,$msgstr,$ref,$flags,$comment,$previous,$automatic)); + } + # delete the apply subroutine + # otherwise it will be redefined. + undef &apply; + return $res; +} + +=item to_utf8() + +Recodes to UTF-8 the PO's msgstrs. Does nothing if the charset is not +specified in the PO file ("CHARSET" value), or if it's already UTF-8 or +ASCII. + +=cut + +sub to_utf8 { + my $this = shift; + my $charset = $this->get_charset(); + + unless ($charset eq "CHARSET" or + $charset =~ /^ascii$/i or + $charset =~ /^utf-8$/i) { + foreach my $msgid ( keys %{$this->{po}} ) { + Encode::from_to($this->{po}{$msgid}{'msgstr'}, $charset, "utf-8"); + } + $this->set_charset("UTF-8"); + } +} + +=back + +=head1 Functions to use a message catalog for translations + +=over 4 + +=item gettext($%) + +Request the translation of the string given as argument in the current catalog. +The function returns the original (untranslated) string if the string was not +found. + +After the string to translate, you can pass a hash of extra +arguments. Here are the valid entries: + +=over + +=item B + +boolean indicating whether we can consider that whitespaces in string are +not important. If yes, the function canonizes the string before looking for +a translation, and wraps the result. + +=item B + +the column at which we should wrap (default: 76). + +=back + +=cut + +sub gettext { + my $self=shift; + my $text=shift; + my (%opt)=@_; + my $res; + + return "" unless defined($text) && length($text); # Avoid returning the header. + my $validoption="reference wrap wrapcol"; + my %validoption; + + map { $validoption{$_}=1 } (split(/ /,$validoption)); + foreach (keys %opt) { + Carp::confess "internal error: unknown arg $_.\n". + "Here are the valid options: $validoption.\n" + unless $validoption{$_}; + } + + $text=canonize($text) + if ($opt{'wrap'}); + + my $esc_text=escape_text($text); + + $self->{gettextqueries}++; + + if ( defined $self->{po}{$esc_text} + and defined $self->{po}{$esc_text}{'msgstr'} + and length $self->{po}{$esc_text}{'msgstr'} + and ( not defined $self->{po}{$esc_text}{'flags'} + or $self->{po}{$esc_text}{'flags'} !~ /fuzzy/)) { + + $self->{gettexthits}++; + $res = unescape_text($self->{po}{$esc_text}{'msgstr'}); + if (defined $self->{po}{$esc_text}{'plural'}) { + if ($self->{po}{$esc_text}{'plural'} eq "0") { + warn wrap_mod("po4a gettextize", dgettext("po4a", + "'%s' is the singular form of a message, ". + "po4a will use the msgstr[0] translation (%s)."), + $esc_text, $res); + } else { + warn wrap_mod("po4a gettextize", dgettext("po4a", + "'%s' is the plural form of a message, ". + "po4a will use the msgstr[1] translation (%s)."), + $esc_text, $res); + } + } + } else { + $res = $text; + } + + if ($opt{'wrap'}) { + if ($self->get_charset =~ /^utf-8$/i) { + $res=Encode::decode_utf8($res); + $res=wrap ($res, $opt{'wrapcol'} || 76); + $res=Encode::encode_utf8($res); + } else { + $res=wrap ($res, $opt{'wrapcol'} || 76); + } + } +# print STDERR "Gettext >>>$text<<<(escaped=$esc_text)=[[[$res]]]\n\n"; + return $res; +} + +=item stats_get() + +Returns statistics about the hit ratio of gettext since the last time that +stats_clear() was called. Please note that it's not the same +statistics than the one printed by msgfmt --statistic. Here, it's statistics +about recent usage of the PO file, while msgfmt reports the status of the +file. Example of use: + + [some use of the PO file to translate stuff] + + ($percent,$hit,$queries) = $pofile->stats_get(); + print "So far, we found translations for $percent\% ($hit of $queries) of strings.\n"; + +=cut + +sub stats_get() { + my $self=shift; + my ($h,$q)=($self->{gettexthits},$self->{gettextqueries}); + my $p = ($q == 0 ? 100 : int($h/$q*10000)/100); + +# $p =~ s/\.00//; +# $p =~ s/(\..)0/$1/; + + return ( $p,$h,$q ); +} + +=item stats_clear() + +Clears the statistics about gettext hits. + +=cut + +sub stats_clear { + my $self = shift; + $self->{gettextqueries} = 0; + $self->{gettexthits} = 0; +} + +=back + +=head1 Functions to build a message catalog + +=over 4 + +=item push(%) + +Push a new entry at the end of the current catalog. The arguments should +form a hash table. The valid keys are: + +=over 4 + +=item B + +the string in original language. + +=item B + +the translation. + +=item B + +an indication of where this string was found. Example: file.c:46 (meaning +in 'file.c' at line 46). It can be a space-separated list in case of +multiple occurrences. + +=item B + +a comment added here manually (by the translators). The format here is free. + +=item B + +a comment which was automatically added by the string extraction +program. See the B<--add-comments> option of the B program for +more information. + +=item B + +space-separated list of all defined flags for this entry. + +Valid flags are: B, B, B, B, B, +B, B, B, B, B, +B, B, B and B. + +See the gettext documentation for their meaning. + +=item B + +this is mostly an internal argument: it is used while gettextizing +documents. The idea here is to parse both the original and the translation +into a PO object, and merge them, using one's msgid as msgid and the +other's msgid as msgstr. To make sure that things get ok, each msgid in PO +objects are given a type, based on their structure (like "chapt", "sect1", +"p" and so on in DocBook). If the types of strings are not the same, that +means that both files do not share the same structure, and the process +reports an error. + +This information is written as automatic comment in the PO file since this +gives to translators some context about the strings to translate. + +=item B + +boolean indicating whether whitespaces can be mangled in cosmetic +reformattings. If true, the string is canonized before use. + +This information is written to the PO file using the B or B flag. + +=item B + +the column at which we should wrap (default: 76). + +This information is not written to the PO file. + +=back + +=cut + +sub push { + my $self=shift; + my %entry=@_; + + my $validoption="wrap wrapcol type msgid msgstr automatic previous flags reference"; + my %validoption; + + map { $validoption{$_}=1 } (split(/ /,$validoption)); + foreach (keys %entry) { + Carp::confess "internal error: unknown arg $_.\n". + "Here are the valid options: $validoption.\n" + unless $validoption{$_}; + } + + unless ($entry{'wrap'}) { + $entry{'flags'} .= " no-wrap"; + } + if (defined ($entry{'msgid'})) { + $entry{'msgid'} = canonize($entry{'msgid'}) + if ($entry{'wrap'}); + + $entry{'msgid'} = escape_text($entry{'msgid'}); + } + if (defined ($entry{'msgstr'})) { + $entry{'msgstr'} = canonize($entry{'msgstr'}) + if ($entry{'wrap'}); + + $entry{'msgstr'} = escape_text($entry{'msgstr'}); + } + + $self->push_raw(%entry); +} + +# The same as push(), but assuming that msgid and msgstr are already escaped +sub push_raw { + my $self=shift; + my %entry=@_; + my ($msgid,$msgstr,$reference,$comment,$automatic,$previous,$flags,$type,$transref)= + ($entry{'msgid'},$entry{'msgstr'}, + $entry{'reference'},$entry{'comment'},$entry{'automatic'}, + $entry{'previous'},$entry{'flags'},$entry{'type'},$entry{'transref'}); + my $keep_conflict = $entry{'conflict'}; + +# print STDERR "Push_raw\n"; +# print STDERR " msgid=>>>$msgid<<<\n" if $msgid; +# print STDERR " msgstr=[[[$msgstr]]]\n" if $msgstr; +# Carp::cluck " flags=$flags\n" if $flags; + + return unless defined($entry{'msgid'}); + + #no msgid => header definition + unless (length($entry{'msgid'})) { +# if (defined($self->{header}) && $self->{header} =~ /\S/) { +# warn dgettext("po4a","Redefinition of the header. ". +# "The old one will be discarded\n"); +# } FIXME: do that iff the header isn't the default one. + $self->{header}=$msgstr; + $self->{header_comment}=$comment; + my $charset = $self->get_charset; + if ($charset ne "CHARSET") { + $self->{encoder}=find_encoding($charset); + } else { + $self->{encoder}=find_encoding("ascii"); + } + return; + } + + if ($self->{options}{'porefs'} =~ m/^none/) { + $reference = ""; + } elsif ($self->{options}{'porefs'} =~ m/^counter/) { + if ($reference =~ m/^(.+?)(?=\S+:\d+)/g) { + my $new_ref = $1; + 1 while $reference =~ s{ # x modifier is added to add formatting and improve readability + \G(\s*)(\S+):\d+ # \G is the last match in m//g (see also the (?=) syntax above) + # $2 is the file name + }{ + $self->{counter}{$2} ||= 0, # each file has its own counter + ++$self->{counter}{$2}, # increment it + $new_ref .= "$1$2:".$self->{counter}{$2} # replace line number by this counter + }gex && pos($reference); + $reference = $new_ref; + } + } elsif ($self->{options}{'porefs'} =~ m/^noline/) { + $reference =~ s/:\d+/:1/g; + } + + if (defined($self->{po}{$msgid})) { + warn wrap_mod("po4a::po", + dgettext("po4a","msgid defined twice: %s"), + $msgid) + if (0); # FIXME: put a verbose stuff + if ( defined $msgstr + and defined $self->{po}{$msgid}{'msgstr'} + and $self->{po}{$msgid}{'msgstr'} ne $msgstr) { + my $txt=quote_text($msgid); + my ($first,$second)= + (format_comment(". ",$self->{po}{$msgid}{'reference'}). + quote_text($self->{po}{$msgid}{'msgstr'}), + + format_comment(". ",$reference). + quote_text($msgstr)); + + if ($keep_conflict) { + if ($self->{po}{$msgid}{'msgstr'} =~ m/^#-#-#-#-# .* #-#-#-#-#\\n/s) { + $msgstr = $self->{po}{$msgid}{'msgstr'}. + "\\n#-#-#-#-# $transref #-#-#-#-#\\n". + $msgstr; + } else { + $msgstr = "#-#-#-#-# ". + $self->{po}{$msgid}{'transref'}. + " #-#-#-#-#\\n". + $self->{po}{$msgid}{'msgstr'}."\\n". + "#-#-#-#-# $transref #-#-#-#-#\\n". + $msgstr; + } + # Every msgid will have the same list of references. + # Only keep the last list. + $self->{po}{$msgid}{'reference'} = ""; + } else { + warn wrap_msg(dgettext("po4a", + "Translations don't match for:\n". + "%s\n". + "-->First translation:\n". + "%s\n". + " Second translation:\n". + "%s\n". + " Old translation discarded."), + $txt,$first,$second); + } + } + } + if (defined $transref) { + $self->{po}{$msgid}{'transref'} = $transref; + } + if (defined($reference) && length($reference)) { + if (defined $self->{po}{$msgid}{'reference'}) { + $self->{po}{$msgid}{'reference'} .= " ".$reference; + } else { + $self->{po}{$msgid}{'reference'} = $reference; + } + } + $self->{po}{$msgid}{'msgstr'} = $msgstr; + $self->{po}{$msgid}{'comment'} = $comment; + $self->{po}{$msgid}{'automatic'} = $automatic; + $self->{po}{$msgid}{'previous'} = $previous; + if (defined($self->{po}{$msgid}{'pos_doc'})) { + $self->{po}{$msgid}{'pos_doc'} .= " ".$self->{count_doc}++; + } else { + $self->{po}{$msgid}{'pos_doc'} = $self->{count_doc}++; + } + unless (defined($self->{po}{$msgid}{'pos'})) { + $self->{po}{$msgid}{'pos'} = $self->{count}++; + } + $self->{po}{$msgid}{'type'} = $type; + $self->{po}{$msgid}{'plural'} = $entry{'plural'} + if defined $entry{'plural'}; + + if (defined($flags)) { + $flags = " $flags "; + $flags =~ s/,/ /g; + foreach my $flag (@known_flags) { + if ($flags =~ /\s$flag\s/) { # if flag to be set + unless ( defined($self->{po}{$msgid}{'flags'}) + && $self->{po}{$msgid}{'flags'} =~ /\b$flag\b/) { + # flag not already set + if (defined $self->{po}{$msgid}{'flags'}) { + $self->{po}{$msgid}{'flags'} .= " ".$flag; + } else { + $self->{po}{$msgid}{'flags'} = $flag; + } + } + } + } + } +# print STDERR "stored ((($msgid)))=>(((".$self->{po}{$msgid}{'msgstr'}.")))\n\n"; + +} + +=back + +=head1 Miscellaneous functions + +=over 4 + +=item count_entries() + +Returns the number of entries in the catalog (without the header). + +=cut + +sub count_entries($) { + my $self=shift; + return $self->{count}; +} + +=item count_entries_doc() + +Returns the number of entries in document. If a string appears multiple times +in the document, it will be counted multiple times + +=cut + +sub count_entries_doc($) { + my $self=shift; + return $self->{count_doc}; +} + +=item msgid($) + +Returns the msgid of the given number. + +=cut + +sub msgid($$) { + my $self=shift; + my $num=shift; + + foreach my $msgid ( keys %{$self->{po}} ) { + return $msgid if ($self->{po}{$msgid}{'pos'} eq $num); + } + return undef; +} + +=item msgid_doc($) + +Returns the msgid with the given position in the document. + +=cut + +sub msgid_doc($$) { + my $self=shift; + my $num=shift; + + foreach my $msgid ( keys %{$self->{po}} ) { + foreach my $pos (split / /, $self->{po}{$msgid}{'pos_doc'}) { + return $msgid if ($pos eq $num); + } + } + return undef; +} + +=item get_charset() + +Returns the character set specified in the PO header. If it hasn't been +set, it will return "CHARSET". + +=cut + +sub get_charset() { + my $self=shift; + + $self->{header} =~ /charset=(.*?)[\s\\]/; + + if (defined $1) { + return $1; + } else { + return "CHARSET"; + } +} + +=item set_charset($) + +This sets the character set of the PO header to the value specified in its +first argument. If you never call this function (and no file with a specified +character set is read), the default value is left to "CHARSET". This value +doesn't change the behavior of this module, it's just used to fill that field +in the header, and to return it in get_charset(). + +=cut + +sub set_charset() { + my $self=shift; + + my ($newchar,$oldchar); + $newchar = shift; + $oldchar = $self->get_charset(); + + $self->{header} =~ s/$oldchar/$newchar/; + $self->{encoder}=find_encoding($newchar); +} + +#----[ helper functions ]--------------------------------------------------- + +# transforme the string from its PO file representation to the form which +# should be used to print it +sub unescape_text { + my $text = shift; + + print STDERR "\nunescape [$text]====" if $debug{'escape'}; + $text = join("",split(/\n/,$text)); + $text =~ s/\\"/"/g; + # unescape newlines + # NOTE on \G: + # The following regular expression introduce newlines. + # Thus, ^ doesn't match all beginnings of lines. + # \G is a zero-width assertion that matches the position + # of the previous substitution with s///g. As every + # substitution ends by a newline, it always matches a + # position just after a newline. + $text =~ s/( # $1: + (\G|[^\\]) # beginning of the line or any char + # different from '\' + (\\\\)* # followed by any even number of '\' + )\\n # and followed by an escaped newline + /$1\n/sgx; # single string, match globally, allow comments + # unescape carriage returns + $text =~ s/( # $1: + (\G|[^\\]) # beginning of the line or any char + # different from '\' + (\\\\)* # followed by any even number of '\' + )\\r # and followed by an escaped carriage return + /$1\r/sgx; # single string, match globally, allow comments + # unescape tabulations + $text =~ s/( # $1: + (\G|[^\\])# beginning of the line or any char + # different from '\' + (\\\\)* # followed by any even number of '\' + )\\t # and followed by an escaped tabulation + /$1\t/mgx; # multilines string, match globally, allow comments + # and unescape the escape character + $text =~ s/\\\\/\\/g; + print STDERR ">$text<\n" if $debug{'escape'}; + + return $text; +} + +# transform the string to its representation as it should be written in PO +# files +sub escape_text { + my $text = shift; + + print STDERR "\nescape [$text]====" if $debug{'escape'}; + $text =~ s/\\/\\\\/g; + $text =~ s/"/\\"/g; + $text =~ s/\n/\\n/g; + $text =~ s/\r/\\r/g; + $text =~ s/\t/\\t/g; + print STDERR ">$text<\n" if $debug{'escape'}; + + return $text; +} + +# put quotes around the string on each lines (without escaping it) +# It does also normalize the text (ie, make sure its representation is wraped +# on the 80th char, but without changing the meaning of the string) +sub quote_text { + my $string = shift; + + return '""' unless defined($string) && length($string); + + print STDERR "\nquote [$string]====" if $debug{'quote'}; + # break lines on newlines, if any + # see unescape_text for an explanation on \G + $string =~ s/( # $1: + (\G|[^\\]) # beginning of the line or any char + # different from '\' + (\\\\)* # followed by any even number of '\' + \\n) # and followed by an escaped newline + /$1\n/sgx; # single string, match globally, allow comments + $string = wrap($string); + my @string = split(/\n/,$string); + $string = join ("\"\n\"",@string); + $string = "\"$string\""; + if (scalar @string > 1 && $string[0] ne '') { + $string = "\"\"\n".$string; + } + + print STDERR ">$string<\n" if $debug{'quote'}; + return $string; +} + +# undo the work of the quote_text function +sub unquote_text { + my $string = shift; + print STDERR "\nunquote [$string]====" if $debug{'quote'}; + $string =~ s/^""\\n//s; + $string =~ s/^"(.*)"$/$1/s; + $string =~ s/"\n"//gm; + # Note: an even number of '\' could precede \\n, but I could not build a + # document to test this + $string =~ s/([^\\])\\n\n/$1!!DUMMYPOPM!!/gm; + $string =~ s|!!DUMMYPOPM!!|\\n|gm; + print STDERR ">$string<\n" if $debug{'quote'}; + return $string; +} + +# canonize the string: write it on only one line, changing consecutive +# whitespace to only one space. +# Warning, it changes the string and should only be called if the string is +# plain text +sub canonize { + my $text=shift; + print STDERR "\ncanonize [$text]====" if $debug{'canonize'}; + $text =~ s/^ *//s; + $text =~ s/^[ \t]+/ /gm; + # if ($text eq "\n"), it messed up the first string (header) + $text =~ s/\n/ /gm if ($text ne "\n"); + $text =~ s/([.)]) +/$1 /gm; + $text =~ s/([^.)]) */$1 /gm; + $text =~ s/ *$//s; + print STDERR ">$text<\n" if $debug{'canonize'}; + return $text; +} + +# wraps the string. We don't use Text::Wrap since it mangles whitespace at +# the end of splited line +sub wrap { + my $text=shift; + return "0" if ($text eq '0'); + my $col=shift || 76; + my @lines=split(/\n/,"$text"); + my $res=""; + my $first=1; + while (defined(my $line=shift @lines)) { + if ($first && length($line) > $col - 10) { + unshift @lines,$line; + $first=0; + next; + } + if (length($line) > $col) { + my $pos=rindex($line," ",$col); + while (substr($line,$pos-1,1) eq '.' && $pos != -1) { + $pos=rindex($line," ",$pos-1); + } + if ($pos == -1) { + # There are no spaces in the first $col chars, pick-up the + # first space + $pos = index($line," "); + } + if ($pos != -1) { + my $end=substr($line,$pos+1); + $line=substr($line,0,$pos+1); + if ($end =~ s/^( +)//) { + $line .= $1; + } + unshift @lines,$end; + } + } + $first=0; + $res.="$line\n"; + } + # Restore the original trailing spaces + $res =~ s/\s+$//s; + if ($text =~ m/(\s+)$/s) { + $res .= $1; + } + return $res; +} + +# outputs properly a '# ... ' line to be put in the PO file +sub format_comment { + my $comment=shift; + my $char=shift; + my $result = "#". $char . $comment; + $result =~ s/\n/\n#$char/gs; + $result =~ s/^#$char$/#/gm; + $result .= "\n"; + return $result; +} + + +1; +__END__ + +=back + +=head1 AUTHORS + + Denis Barbier + Martin Quinson (mquinson#debian.org) + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Ini.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Ini.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Ini.pm (revision 2840) @@ -0,0 +1,119 @@ +# Locale::Po4a::Ini -- Convert ini files to PO file, for translation. +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING). +# + +############################################################################ +# Modules and declarations +############################################################################ + +use Locale::Po4a::TransTractor qw(process new); +use Locale::Po4a::Common; + +package Locale::Po4a::Ini; + +use 5.006; +use strict; +use warnings; + +require Exporter; + +use vars qw(@ISA @EXPORT $AUTOLOAD); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +my $debug=0; + +sub initialize {} + + +sub parse { + my $self=shift; + my ($line,$ref); + my $par; + + LINE: + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { + chomp($line); + print STDERR "begin\n" if $debug; + + if ($line =~ /\"/) { + print STDERR "Start of line containing \".\n" if $debug; + # Text before the first quote + $line =~ m/(^[^"\r\n]*")/; + my $pre_text = $1; + print STDERR " PreText=".$pre_text."\n" if $debug; + # The text for translation + $line =~ m/("[^\r\n]*")/; + my $quoted_text = $1; + print STDERR " QuotedText=".$quoted_text."\n" if $debug; + # Text after last quote + $line =~ m/("[^"\n]*$)/; + my $post_text = $1; + print STDERR " PostText=".$post_text."\n" if $debug; + # Remove starting and ending quotes from the translation text, if any + $quoted_text =~ s/^"//g; + $quoted_text =~ s/"$//g; + # Translate the string it + $par = $self->translate($quoted_text, $ref); + # Escape the \n characters + $par =~ s/\n/\\n/g; + # Now push the result + $self->pushline($pre_text.$par.$post_text."\n"); + print STDERR "End of line containing \".\n" if $debug; + } + else + { + print STDERR "Other stuff\n" if $debug; + $self->pushline("$line\n"); + } + # Reinit the loop + ($line,$ref)=$self->shiftline(); + } +} + +############################################################################## +# Module return value and documentation +############################################################################## + +1; +__END__ + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Ini - convert INI files from/to PO files + +=head1 DESCRIPTION + +Locale::Po4a::Ini is a module to help the translation of INI files into other +[human] languages. + +The module searches for lines of the following format and extracts the quoted +text: + +identificator="text than can be translated" + +NOTE: If the text is not quoted, it will be ignored. + +=head1 SEE ALSO + +L, L + +=head1 AUTHORS + + Razvan Rusu + Costin Stroie + +=head1 COPYRIGHT AND LICENSE + +Copyright 2006 by BitDefender + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/NewsDebian.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/NewsDebian.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/NewsDebian.pm (revision 2840) @@ -0,0 +1,154 @@ +#!/usr/bin/perl -w + +# Po4a::NewsDebian.pm +# +# extract and translate translatable strings from a NEWS.Debian documents +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::NewsDebian - convert NEWS.Debian documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::NewsDebian is a module to help the translation of the +NEWS.Debian files into other [human] languages. Those files are where +maintainer are supposed to write the important news about their package. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +NONE. + +=head1 STATUS OF THIS MODULE + +Not tested. + +A finer split of the entries may be preferable (search for /^ */, for +example), but this version is more robust and NEWS.Debian entries are not +supposed to change that often. + +=cut + +package Locale::Po4a::NewsDebian; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + + +sub initialize {} + +sub parse { + my $self = shift; + + my ($blanklines)=(""); # We want to preserve the blank lines inside the entry, and strip the extrem ones + + my ($body)=""; # the accumulated paragraph + my ($bodyref)=""; + my ($bodytype)=""; + + my ($line,$lref); + + # main loop + ($line,$lref)=$self->shiftline(); + print "seen >>$line<<\n" if $self->debug(); + while (defined($line)) { + + # Begining of an entry + if ($line =~ m/^(\w[-+0-9a-z.]*) \(([^\(\) \t]+)\)((\s+[-0-9a-z]+)+)\;/i) { + + die wrap_ref_mod($lref, "po4a::newsdebian", dgettext("po4a", "Begin of a new entry before the end of previous one")) + if (length ($body)); + + $self->pushline($line."\n"); + + # Signature of this entry + $bodyref = $lref; + $bodytype = $line; + + # eat all leading empty lines + ($line,$lref)=$self->shiftline(); + while (defined($line) && $line =~ m/^\s*$/) { + print "Eat >>$line<<\n" if $self->debug(); + ($line,$lref)=$self->shiftline(); + } + # ups, ate one line too much. Put it back. + $self->unshiftline($line,$lref); + + + # get ready to read the entry (cleanups) + $blanklines = ""; + + # End of current entry + } elsif ($line =~ m/^ \-\- (.*) <(.*)> .*$/) { #((\w+\,\s*)?\d{1,2}\s+\w+\s+\d{4}\s+\d{1,2}:\d\d:\d\d\s+[-+]\d{4}(\s+\([^\\\(\)]\))?) *$/) { + + $self->translate($body, $bodyref, $bodytype, + wrap=>0); + $body=""; + + # non-specific line + } else { + + if ($line =~ /^\s*$/) { + $blanklines .= "$line"; + } else { + $body .= $blanklines.$line; + $blanklines = ""; + } + } + + ($line,$lref)=$self->shiftline(); + print "seen >>".($line || '')."<<\n" if $self->debug(); + } +} + +1; + +=head1 AUTHORS + +This module is loosely inspired from /usr/lib/dpkg/parsechangelog/debian, which is: + + Copyright (C) 1996 Ian Jackson. This is free software; see the GNU + General Public Licence version 2 or later for copying conditions. There + is NO warranty. + +The adaptation for po4a was done by: + + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + + Copyright (c) 1996 by Ian Jackson. + Copyright 2005 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Prop.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Prop.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Prop.pm (revision 2840) @@ -0,0 +1,133 @@ +# Locale::Po4a::Prop -- Convert Java property and OSX/NeXTSTEP strings files +# to PO file, for translation. +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING). +# + +############################################################################ +# Modules and declarations +############################################################################ + +package Locale::Po4a::Prop; + +use 5.006; +use strict; +use warnings; + +require Exporter; + +use vars qw(@ISA @EXPORT $AUTOLOAD); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +my $debug=0; + +sub initialize { + my $self = shift; + my %options = @_; + + $self->{options}{'wrap'}=0; + + foreach my $opt (keys %options) { + if ($options{$opt}) { + die wrap_mod("po4a::prop", + dgettext("po4a", "Unknown option: %s"), $opt) + unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options{$opt}; + } + } +} +sub parse { + my $self=shift; + my ($line,$ref); + my $par; + + LINE: + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { + chomp($line); + print STDERR "begin:$line\n" if $debug; + + if ($line =~ m/("[^"]*")[^=]*=[^"]*"(.*)/) { # " + my @paragraph = (); + my $pre_text = $1; + print STDERR " PreText=".$pre_text."\n" if $debug; + my $quoted_text = $2; + # The text for translation + print STDERR " QuotedText=".$quoted_text."\n" if $debug; + if ($quoted_text !~ /";[^"]*/) { + @paragraph = ("$quoted_text\n"); + do { + ($line, undef) = $self->shiftline(); + push @paragraph, $line; + } while ($line !~ /";[^"]*/); + } + if (@paragraph) { + $quoted_text = join('', @paragraph); + } + + # Remove the final "; + $quoted_text =~ s/"\s*;[^"]*$//; + # Translate the string + $par = $self->translate($quoted_text, $ref, $pre_text, + 'wrap'=>$self->{options}{'wrap'}); + # Now push the result + $self->pushline($pre_text .' = "'.$par."\";\n"); + print STDERR "End of line containing \".\n" if $debug; + } + else + { + print STDERR "Other stuff\n" if $debug; + $self->pushline("$line\n"); + } + # Reinit the loop + ($line,$ref)=$self->shiftline(); + } +} + +############################################################################## +# Module return value and documentation +############################################################################## + +1; +__END__ + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Prop - convert Java property and OSX/NeXTSTEP strings from/to PO files + +=head1 DESCRIPTION + +Locale::Po4a::Prop is a module to help the translation of Java property and +OSX/NeXTSTEP strings files into other [human] languages. + +The module searches for lines of the following format and extracts the quoted +text: + +"identificator"="text than can be translated"; + +NOTE: If the text is not quoted, it will be ignored. + +=head1 SEE ALSO + +L, L + +=head1 AUTHORS + + Yves Blusseau + +=head1 COPYRIGHT AND LICENSE + +Copyright 2012 by Yves Blusseau + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Docbook.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Docbook.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Docbook.pm (revision 2840) @@ -0,0 +1,2042 @@ +#!/usr/bin/perl +# aptitude: cmdsynopsis => missing removal of leading spaces + +# Po4a::Docbook.pm +# +# extract and translate translatable strings from DocBook XML documents. +# +# This code extracts plain text from tags and attributes on DocBook XML +# documents. +# +# Copyright (c) 2004 by Jordi Vilalta +# Copyright (c) 2007-2009 by Nicolas François +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Docbook - convert DocBook XML documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Docbook is a module to help the translation of DocBook XML +documents into other [human] languages. + +Please note that this module is still under heavy development, and not +distributed in official po4a release since we don't feel it to be mature +enough. If you insist on trying, check the CVS out. + +=head1 STATUS OF THIS MODULE + +This module is fully functional, as it relies in the L +module. This only defines the translatable tags and attributes. + +The only known issue is that it doesn't handle entities yet, and this includes +the file inclusion entities, but you can translate most of those files alone +(except the typical entities files), and it's usually better to maintain them +separated. + +=head1 SEE ALSO + +L, L, L + +=head1 AUTHORS + + Jordi Vilalta + +=head1 COPYRIGHT AND LICENSE + + Copyright (c) 2004 by Jordi Vilalta + Copyright (c) 2007-2009 by Nicolas François + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +package Locale::Po4a::Docbook; + +use 5.006; +use strict; +use warnings; + +use Locale::Po4a::Xml; + +use vars qw(@ISA); +@ISA = qw(Locale::Po4a::Xml); + +sub initialize { + my $self = shift; + my %options = @_; + + $self->SUPER::initialize(%options); + $self->{options}{'wrap'}=1; + $self->{options}{'doctype'}=$self->{options}{'doctype'} || 'docbook xml'; + +# AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA + + # abbrev; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # abstract; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # accel; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # ackno; does not contain text; Formatted as a displayed block + # Replaced by acknowledgements in DocBook v5.0 + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + # acknowledgements; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # acronym; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # action; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # address; contains text; Formatted as a displayed block; verbatim + $self->{options}{'_default_translated'} .= " W
"; + $self->{options}{'_default_placeholder'} .= "
"; + + # affiliation; does not contain text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # alt; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # anchor; does not contain text; Produces no output + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # annotation; does not contain text; + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # answer; does not contain text; + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # appendix; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # appendixinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # application; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # arc; does not contain text; + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # area; does not contain text; + # NOTE: the area is not translatable as is, but the coords + # attribute might be. + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # areaset; does not contain text; + # NOTE: the areaset is not translatable as is. depending on the + # language there might be more or less area tags inside. + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # areaspec; does not contain text; + # NOTE: see area and areaset + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # arg; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # artheader; does not contain text; renamed to articleinfo in v4.0 + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # article; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= "
"; + $self->{options}{'_default_break'} .= "
"; + + # articleinfo; does not contain text; v4 only + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # artpagenums; contains text; Formatted inline + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # attribution; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # audiodata; does not contain text; + # NOTE: the attributes might be translated + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + $self->{options}{'_default_attributes'}.=' fileref'; + + # audioobject; does not contain text; + # NOTE: might be contained in a inlinemediaobject + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # author; does not contain text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # authorblurb; does not contain text; Formatted as a displayed block. + # v4, not in v5 + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # authorgroup; does not contain text; Formatted inline or as a + # displayed block depending on context + # NOTE: given the possible parents, it is probably very rarely + # inlined + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # authorinitials; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + +# BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB + + # beginpage; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # bibliocoverage; contains text; Formatted inline + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # bibliodiv; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # biblioentry; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # bibliography; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # bibliographyinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # biblioid; contains text; Formatted inline + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # bibliolist; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " "; + $self->{options}{'_default_break'} .= " "; + + # bibliomisc; contains text; Formatted inline + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_inline'} .= " "; + + # bibliomixed; contains text; Formatted as a displayed block + $self->{options}{'_default_translated'} .= " "; + $self->{options}{'_default_placeholder'} .= " "; + + # bibliomset; contains text; Formatted as a displayed block + # NOTE: content might need to be inlined, e.g. + $self->{options}{'_default_translated'} .= " <bibliomset>"; + $self->{options}{'_default_placeholder'} .= " <bibliomset>"; + + # biblioref; does not contain text; Formatted inline + $self->{options}{'_default_untranslated'} .= " <biblioref>"; + $self->{options}{'_default_inline'} .= " <biblioref>"; + + # bibliorelation; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <bibliorelation>"; + $self->{options}{'_default_inline'} .= " <bibliorelation>"; + + # biblioset; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <biblioset>"; + $self->{options}{'_default_break'} .= " <biblioset>"; + + # bibliosource; contains text; Formatted inline + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <bibliosource>"; + $self->{options}{'_default_inline'} .= " <bibliosource>"; + + # blockinfo; does not contain text; v4.2, not in v5 + $self->{options}{'_default_untranslated'} .= " <blockinfo>"; + $self->{options}{'_default_placeholder'} .= " <blockinfo>"; + + # blockquote; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <blockquote>"; + $self->{options}{'_default_break'} .= " <blockquote>"; + + # book; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <book>"; + $self->{options}{'_default_break'} .= " <book>"; + + # bookbiblio; does not contain text; Formatted as a displayed block + # Removed in v4.0 + $self->{options}{'_default_untranslated'} .= " <bookbiblio>"; + $self->{options}{'_default_break'} .= " <bookbiblio>"; + + # bookinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <bookinfo>"; + $self->{options}{'_default_placeholder'} .= " <bookinfo>"; + + # bridgehead; contains text; Formatted as a displayed block + $self->{options}{'_default_translated'} .= " <bridgehead>"; + $self->{options}{'_default_break'} .= " <bridgehead>"; + +# CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC + + # callout; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <callout>"; + $self->{options}{'_default_break'} .= " <callout>"; + + # calloutlist; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <calloutlist>"; + $self->{options}{'_default_break'} .= " <calloutlist>"; + + # caption; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <caption>"; + $self->{options}{'_default_break'} .= " <caption>"; + + # caption (db.html.caption); contains text; Formatted as a displayed block + # TODO: Check if this works + $self->{options}{'_default_translated'} .= " <table><caption>"; + $self->{options}{'_default_break'} .= " <table><caption>"; + + # caution; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <caution>"; + $self->{options}{'_default_break'} .= " <caution>"; + + # chapter; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <chapter>"; + $self->{options}{'_default_break'} .= " <chapter>"; + + # chapterinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <chapterinfo>"; + $self->{options}{'_default_placeholder'} .= " <chapterinfo>"; + + # citation; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <citation>"; + $self->{options}{'_default_inline'} .= " <citation>"; + + # citebiblioid; contains text; Formatted inline + # NOTE: maybe untranslated? + $self->{options}{'_default_translated'} .= " <citebiblioid>"; + $self->{options}{'_default_inline'} .= " <citebiblioid>"; + + # citerefentry; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <citerefentry>"; + $self->{options}{'_default_inline'} .= " <citerefentry>"; + + # citetitle; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <citetitle>"; + $self->{options}{'_default_inline'} .= " <citetitle>"; + + # city; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <city>"; + $self->{options}{'_default_inline'} .= " <city>"; + + # classname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <classname>"; + $self->{options}{'_default_inline'} .= " <classname>"; + + # classsynopsis; does not contain text; may be in a para + # NOTE: It may contain a classsynopsisinfo, which should be + # verbatim + # XXX: since it is in untranslated class, does the W flag takes + # effect? + $self->{options}{'_default_untranslated'} .= " W<classsynopsis>"; + $self->{options}{'_default_placeholder'} .= " <classsynopsis>"; + + # classsynopsisinfo; contains text; + # NOTE: see above + $self->{options}{'_default_translated'} .= " W<classsynopsisinfo>"; + $self->{options}{'_default_inline'} .= " <classsynopsisinfo>"; + + # cmdsynopsis; does not contain text; may be in a para + # NOTE: It may be clearer as a verbatim block + # XXX: since it is in untranslated class, does the W flag takes + # effect? => not completely. Rewrap afterward? + $self->{options}{'_default_untranslated'} .= " W<cmdsynopsis>"; + $self->{options}{'_default_placeholder'} .= " <cmdsynopsis>"; + + # co; does not contain text; Formatted inline + # XXX: tranlsated or not? (label attribute) + $self->{options}{'_default_translated'} .= " <co>"; + $self->{options}{'_default_inline'} .= " <co>"; + + # code; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <code>"; + $self->{options}{'_default_inline'} .= " <code>"; + + # col; does not contain text; + # NOTE: could be translated to change the layout in a translation + # To be done on colgroup in that case. + $self->{options}{'_default_untranslated'} .= " <col>"; + $self->{options}{'_default_break'} .= " <col>"; + + # colgroup; does not contain text; + # NOTE: could be translated to change the layout in a translation + $self->{options}{'_default_untranslated'} .= " <colgroup>"; + $self->{options}{'_default_break'} .= " <colgroup>"; + + # collab; does not contain text; Formatted inline or as a + # displayed block depending on context + # NOTE: could be in the break class + $self->{options}{'_default_untranslated'} .= " <collab>"; + $self->{options}{'_default_inline'} .= " <collab>"; + + # collabname; contains text; Formatted inline or as a + # displayed block depending on context; v4, not in v5 + $self->{options}{'_default_translated'} .= " <collabname>"; + $self->{options}{'_default_inline'} .= " <collabname>"; + + # colophon; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <colophon>"; + $self->{options}{'_default_break'} .= " <colophon>"; + + # colspec; does not contain text; + # NOTE: could be translated to change the layout in a translation + $self->{options}{'_default_untranslated'} .= " <colspec>"; + $self->{options}{'_default_break'} .= " <colspec>"; + + # command; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <command>"; + $self->{options}{'_default_inline'} .= " <command>"; + + # comment; contains text; Formatted inline or as a displayed block + # Renamed to remark in v4.0 + $self->{options}{'_default_translated'} .= " <comment>"; + $self->{options}{'_default_inline'} .= " <comment>"; + + # computeroutput; contains text; Formatted inline + # NOTE: "is not a verbatim environment, but an inline." + $self->{options}{'_default_translated'} .= " <computeroutput>"; + $self->{options}{'_default_inline'} .= " <computeroutput>"; + + # confdates; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " <confdates>"; + $self->{options}{'_default_inline'} .= " <confdates>"; + + # confgroup; does not contain text; Formatted inline or as a + # displayed block depending on context + # NOTE: could be in the break class + $self->{options}{'_default_untranslated'} .= " <confgroup>"; + $self->{options}{'_default_inline'} .= " <confgroup>"; + + # confnum; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " <confnum>"; + $self->{options}{'_default_inline'} .= " <confnum>"; + + # confsponsor; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " <confsponsor>"; + $self->{options}{'_default_inline'} .= " <confsponsor>"; + + # conftitle; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " <conftitle>"; + $self->{options}{'_default_inline'} .= " <conftitle>"; + + # constant; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <constant>"; + $self->{options}{'_default_inline'} .= " <constant>"; + + # constraint; does not contain text; + # NOTE: it might be better to have the production as verbatim + # Keeping the constrainst inline to have it close to the + # lhs or rhs. + # The attribute is translatable + $self->{options}{'_default_untranslated'} .= " <constraint>"; + $self->{options}{'_default_break'} .= " <constraint>"; + + # constraintdef; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <constraintdef>"; + $self->{options}{'_default_break'} .= " <constraintdef>"; + + # constructorsynopsis; does not contain text; may be in a para + # NOTE: It may be clearer as a verbatim block + # XXX: since it is in untranslated class, does the W flag takes + # effect? + $self->{options}{'_default_untranslated'} .= " W<constructorsynopsis>"; + $self->{options}{'_default_placeholder'} .= " <constructorsynopsis>"; + + # contractnum; contains text; Formatted inline or as a displayed block + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <contractnum>"; + $self->{options}{'_default_inline'} .= " <contractnum>"; + + # contractsponsor; contains text; Formatted inline or as a displayed block + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <contractsponsor>"; + $self->{options}{'_default_inline'} .= " <contractsponsor>"; + + # contrib; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_translated'} .= " <contrib>"; + $self->{options}{'_default_inline'} .= " <contrib>"; + + # copyright; contains text; Formatted inline or as a displayed block + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <copyright>"; + $self->{options}{'_default_inline'} .= " <copyright>"; + + # coref; does not contain text; Formatted inline + # XXX: tranlsated or not? (label attribute) + $self->{options}{'_default_translated'} .= " <coref>"; + $self->{options}{'_default_inline'} .= " <coref>"; + + # corpauthor; contains text; Formatted inline or as a + # displayed block depending on context; v4, not in v5 + $self->{options}{'_default_translated'} .= " <corpauthor>"; + $self->{options}{'_default_inline'} .= " <corpauthor>"; + + # corpcredit; contains text; Formatted inline or as a + # displayed block depending on context; v4, not in v5 + $self->{options}{'_default_translated'} .= " <corpcredit>"; + $self->{options}{'_default_inline'} .= " <corpcredit>"; + + # corpname; contains text; Formatted inline or as a + # displayed block depending on context; v4, not in v5 + $self->{options}{'_default_translated'} .= " <corpname>"; + $self->{options}{'_default_inline'} .= " <corpname>"; + + # country; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <country>"; + $self->{options}{'_default_inline'} .= " <country>"; + + # cover; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <cover>"; + $self->{options}{'_default_break'} .= " <cover>"; + +# DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD + + # database; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <database>"; + $self->{options}{'_default_inline'} .= " <database>"; + + # date; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <date>"; + $self->{options}{'_default_inline'} .= " <date>"; + + # dedication; contains text; Formatted as a displayed block + $self->{options}{'_default_translated'} .= " <dedication>"; + $self->{options}{'_default_break'} .= " <dedication>"; + + # destructorsynopsis; does not contain text; may be in a para + # NOTE: It may be clearer as a verbatim block + # XXX: since it is in untranslated class, does the W flag takes + # effect? + $self->{options}{'_default_untranslated'} .= " W<destructorsynopsis>"; + $self->{options}{'_default_placeholder'} .= " <destructorsynopsis>"; + + # docinfo; does not contain text; removed in v4.0 + $self->{options}{'_default_untranslated'} .= " <docinfo>"; + $self->{options}{'_default_placeholder'} .= " <docinfo>"; + +# EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE + + # edition; contains text; Formatted inline or as a displayed block + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <edition>"; + $self->{options}{'_default_inline'} .= " <edition>"; + + # editor; does not contain text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_untranslated'} .= " <editor>"; + $self->{options}{'_default_inline'} .= " <editor>"; + + # email; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <email>"; + $self->{options}{'_default_inline'} .= " <email>"; + + # emphasis; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <emphasis>"; + $self->{options}{'_default_inline'} .= " <emphasis>"; + + # entry; contains text; + $self->{options}{'_default_translated'} .= " <entry>"; + $self->{options}{'_default_break'} .= " <entry>"; + + # entrytbl; does not contain text; + $self->{options}{'_default_untranslated'} .= " <entrytbl>"; + $self->{options}{'_default_break'} .= " <entrytbl>"; + + # envar; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <envar>"; + $self->{options}{'_default_inline'} .= " <envar>"; + + # epigraph; contains text; Formatted as a displayed block. + # NOTE: maybe contained in a para + $self->{options}{'_default_translated'} .= " <epigraph>"; + $self->{options}{'_default_placeholder'} .= " <epigraph>"; + + # equation; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <equation>"; + $self->{options}{'_default_break'} .= " <equation>"; + + # errorcode; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <errorcode>"; + $self->{options}{'_default_inline'} .= " <errorcode>"; + + # errorname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <errorname>"; + $self->{options}{'_default_inline'} .= " <errorname>"; + + # errortext; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <errortext>"; + $self->{options}{'_default_inline'} .= " <errortext>"; + + # errortype; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <errortype>"; + $self->{options}{'_default_inline'} .= " <errortype>"; + + # example; does not contain text; Formatted as a displayed block. + # NOTE: maybe contained in a para + $self->{options}{'_default_untranslated'} .= " <example>"; + $self->{options}{'_default_placeholder'} .= " <example>"; + + # exceptionname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <exceptionname>"; + $self->{options}{'_default_inline'} .= " <exceptionname>"; + + # extendedlink; does not contain text; + $self->{options}{'_default_untranslated'} .= " <extendedlink>"; + $self->{options}{'_default_inline'} .= " <extendedlink>"; + +# FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF + + # fax; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <fax>"; + $self->{options}{'_default_inline'} .= " <fax>"; + + # fieldsynopsis; does not contain text; may be in a para + $self->{options}{'_default_untranslated'} .= " <fieldsynopsis>"; + $self->{options}{'_default_inline'} .= " <fieldsynopsis>"; + + # figure; does not contain text; Formatted as a displayed block. + # NOTE: maybe contained in a para + $self->{options}{'_default_untranslated'} .= " <figure>"; + $self->{options}{'_default_placeholder'} .= " <figure>"; + + # filename; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <filename>"; + $self->{options}{'_default_inline'} .= " <filename>"; + + # firstname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <firstname>"; + $self->{options}{'_default_inline'} .= " <firstname>"; + + # firstterm; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <firstterm>"; + $self->{options}{'_default_inline'} .= " <firstterm>"; + + # footnote; contains text; + $self->{options}{'_default_translated'} .= " <footnote>"; + $self->{options}{'_default_placeholder'} .= " <footnote>"; + + # footnoteref; contains text; + $self->{options}{'_default_translated'} .= " <footnoteref>"; + $self->{options}{'_default_inline'} .= " <footnoteref>"; + + # foreignphrase; contains text; + $self->{options}{'_default_translated'} .= " <foreignphrase>"; + $self->{options}{'_default_inline'} .= " <foreignphrase>"; + + # formalpara; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <formalpara>"; + $self->{options}{'_default_break'} .= " <formalpara>"; + + # funcdef; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <funcdef>"; + $self->{options}{'_default_inline'} .= " <funcdef>"; + + # funcparams; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <funcparams>"; + $self->{options}{'_default_inline'} .= " <funcparams>"; + + # funcprototype; does not contain text; + # NOTE: maybe contained in a funcsynopsis, contained in a para + $self->{options}{'_default_untranslated'} .= " <funcprototype>"; + $self->{options}{'_default_placeholder'} .= " <funcprototype>"; + + # funcsynopsis; does not contain text; + # NOTE: maybe contained in a para + $self->{options}{'_default_untranslated'} .= " <funcsynopsis>"; + $self->{options}{'_default_placeholder'} .= " <funcsynopsis>"; + + # funcsynopsisinfo; contains text; verbatim + # NOTE: maybe contained in a funcsynopsis, contained in a para + $self->{options}{'_default_translated'} .= " W<funcsynopsisinfo>"; + $self->{options}{'_default_placeholder'} .= " <funcsynopsisinfo>"; + + # function; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <function>"; + $self->{options}{'_default_inline'} .= " <function>"; + +# GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG + + # glossary; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <glossary>"; + $self->{options}{'_default_break'} .= " <glossary>"; + + # glossaryinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <glossaryinfo>"; + $self->{options}{'_default_placeholder'} .= " <glossaryinfo>"; + + # glossdef; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <glossdef>"; + $self->{options}{'_default_break'} .= " <glossdef>"; + + # glossdiv; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <glossdiv>"; + $self->{options}{'_default_break'} .= " <glossdiv>"; + + # glossentry; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <glossentry>"; + $self->{options}{'_default_break'} .= " <glossentry>"; + + # glosslist; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <glosslist>"; + $self->{options}{'_default_break'} .= " <glosslist>"; + + # glosssee; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <glosssee>"; + $self->{options}{'_default_break'} .= " <glosssee>"; + + # glossseealso; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <glossseealso>"; + $self->{options}{'_default_break'} .= " <glossseealso>"; + + # glossterm; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <glossterm>"; + $self->{options}{'_default_inline'} .= " <glossterm>"; + + # graphic; does not contain text; Formatted as a displayed block + # v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <graphic>"; + $self->{options}{'_default_inline'} .= " <graphic>"; + $self->{options}{'_default_attributes'}.=' <graphic>fileref'; + + # graphicco; does not contain text; Formatted as a displayed block. + # v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <graphicco>"; + $self->{options}{'_default_placeholder'} .= " <graphicco>"; + + # group; does not contain text; Formatted inline + $self->{options}{'_default_untranslated'} .= " W<group>"; + $self->{options}{'_default_inline'} .= " <group>"; + + # guibutton; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <guibutton>"; + $self->{options}{'_default_inline'} .= " <guibutton>"; + + # guiicon; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <guiicon>"; + $self->{options}{'_default_inline'} .= " <guiicon>"; + + # guilabel; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <guilabel>"; + $self->{options}{'_default_inline'} .= " <guilabel>"; + + # guimenu; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <guimenu>"; + $self->{options}{'_default_inline'} .= " <guimenu>"; + + # guimenuitem; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <guimenuitem>"; + $self->{options}{'_default_inline'} .= " <guimenuitem>"; + + # guisubmenu; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <guisubmenu>"; + $self->{options}{'_default_inline'} .= " <guisubmenu>"; + +# HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH + + # hardware; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <hardware>"; + $self->{options}{'_default_inline'} .= " <hardware>"; + + # highlights; does not contain text; Formatted inline + # v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <highlights>"; + $self->{options}{'_default_break'} .= " <highlights>"; + + # holder; contains text; + # NOTE: may depend on the copyright container + $self->{options}{'_default_translated'} .= " <holder>"; + $self->{options}{'_default_inline'} .= " <holder>"; + + # honorific; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <honorific>"; + $self->{options}{'_default_inline'} .= " <honorific>"; + + # html:button; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <html:button>"; + $self->{options}{'_default_inline'} .= " <html:button>"; + + # html:fieldset; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <html:fieldset>"; + $self->{options}{'_default_inline'} .= " <html:fieldset>"; + + # html:form; does not contain text; + $self->{options}{'_default_translated'} .= " <html:form>"; + $self->{options}{'_default_inline'} .= " <html:form>"; + + # html:input; does not contain text; Formatted inline + # NOTE: attributes are translatable + $self->{options}{'_default_translated'} .= " <html:input>"; + $self->{options}{'_default_inline'} .= " <html:input>"; + + # html:label; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <html:label>"; + $self->{options}{'_default_inline'} .= " <html:label>"; + + # html:legend; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <html:legend>"; + $self->{options}{'_default_inline'} .= " <html:legend>"; + + # html:option; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <html:option>"; + $self->{options}{'_default_inline'} .= " <html:option>"; + + # html:select; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <html:select>"; + $self->{options}{'_default_inline'} .= " <html:select>"; + + # html:textarea; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <html:textarea>"; + $self->{options}{'_default_placeholder'} .= " <html:textarea>"; + + # imagedata; does not contain text; May be formatted inline or + # as a displayed block, depending on context + $self->{options}{'_default_translated'} .= " <imagedata>"; + $self->{options}{'_default_inline'} .= " <imagedata>"; + $self->{options}{'_default_attributes'}.=' <imagedata>fileref'; + + # imageobject; does not contain text; May be formatted inline or + # as a displayed block, depending on context + $self->{options}{'_default_untranslated'} .= " <imageobject>"; + $self->{options}{'_default_inline'} .= " <imageobject>"; + + # imageobjectco; does not contain text; Formatted as a displayed block + # NOTE: may be in a inlinemediaobject + # TODO: check if this works when the inlinemediaobject is defined + # as inline + $self->{options}{'_default_untranslated'} .= " <imageobjectco>"; + $self->{options}{'_default_break'} .= " <imageobjectco>"; + + # important; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <important>"; + $self->{options}{'_default_break'} .= " <important>"; + + # index; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <index>"; + $self->{options}{'_default_break'} .= " <index>"; + + # indexdiv; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <indexdiv>"; + $self->{options}{'_default_break'} .= " <indexdiv>"; + + # indexentry; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <indexentry>"; + $self->{options}{'_default_break'} .= " <indexentry>"; + + # indexinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <indexinfo>"; + $self->{options}{'_default_placeholder'} .= " <indexinfo>"; + + # indexterm; does not contain text; + $self->{options}{'_default_untranslated'} .= " <indexterm>"; + $self->{options}{'_default_placeholder'} .= " <indexterm>"; + + # info; does not contain text; + $self->{options}{'_default_untranslated'} .= " <info>"; + $self->{options}{'_default_placeholder'} .= " <info>"; + + # informalequation; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <informalequation>"; + $self->{options}{'_default_placeholder'} .= " <informalequation>"; + + # informalexample; does not contain text; Formatted as a displayed block. + # NOTE: can be in a para + $self->{options}{'_default_untranslated'} .= " <informalexample>"; + $self->{options}{'_default_break'} .= " <informalexample>"; + + # informalfigure; does not contain text; Formatted as a displayed block. + # NOTE: can be in a para + $self->{options}{'_default_untranslated'} .= " <informalfigure>"; + $self->{options}{'_default_break'} .= " <informalfigure>"; + + # informaltable; does not contain text; Formatted as a displayed block. + # NOTE: can be in a para + $self->{options}{'_default_untranslated'} .= " <informaltable>"; + $self->{options}{'_default_break'} .= " <informaltable>"; + + # initializer; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <initializer>"; + $self->{options}{'_default_inline'} .= " <initializer>"; + + # inlineequation; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " W<inlineequation>"; + $self->{options}{'_default_placeholder'} .= " <inlineequation>"; + + # inlinegraphic; does not contain text; Formatted inline + # empty; v4, not in v5 + $self->{options}{'_default_translated'} .= " W<inlinegraphic>"; + $self->{options}{'_default_inline'} .= " <inlinegraphic>"; + + # inlinemediaobject; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <inlinemediaobject>"; + $self->{options}{'_default_placeholder'} .= " <inlinemediaobject>"; + + # interface; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <interface>"; + $self->{options}{'_default_inline'} .= " <interface>"; + + # interfacedefinition; contains text; Formatted inline + # Removed in v4.0 + $self->{options}{'_default_translated'} .= " <interfacedefinition>"; + $self->{options}{'_default_inline'} .= " <interfacedefinition>"; + + # interfacename; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <interfacename>"; + $self->{options}{'_default_inline'} .= " <interfacename>"; + + # invpartnumber; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <invpartnumber>"; + $self->{options}{'_default_inline'} .= " <invpartnumber>"; + + # isbn; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <isbn>"; + $self->{options}{'_default_inline'} .= " <isbn>"; + + # issn; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <issn>"; + $self->{options}{'_default_inline'} .= " <issn>"; + + # issuenum; contains text; Formatted inline or as a displayed block + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <issuenum>"; + $self->{options}{'_default_inline'} .= " <issuenum>"; + + # itemizedlist; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <itemizedlist>"; + $self->{options}{'_default_break'} .= " <itemizedlist>"; + + # itermset; does not contain text; + # FIXME + $self->{options}{'_default_untranslated'} .= " <itermset>"; + $self->{options}{'_default_inline'} .= " <itermset>"; + +# JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ + + # jobtitle; contains text; Formatted inline or as a displayed block + # NOTE: can be in a para + $self->{options}{'_default_translated'} .= " <jobtitle>"; + $self->{options}{'_default_inline'} .= " <jobtitle>"; + +# KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK + + # keycap; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <keycap>"; + $self->{options}{'_default_inline'} .= " <keycap>"; + + # keycode; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <keycode>"; + $self->{options}{'_default_inline'} .= " <keycode>"; + + # keycombo; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <keycombo>"; + $self->{options}{'_default_inline'} .= " <keycombo>"; + + # keysym; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <keysym>"; + $self->{options}{'_default_inline'} .= " <keysym>"; + + # keyword; contains text; + # NOTE: could be inline + $self->{options}{'_default_translated'} .= " <keyword>"; + $self->{options}{'_default_break'} .= " <keyword>"; + + # keywordset; contains text; Formatted inline or as a displayed block + # NOTE: could be placeholder/break + $self->{options}{'_default_translated'} .= " <keywordset>"; + $self->{options}{'_default_break'} .= " <keywordset>"; + +# LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL + + # label; contains text; Formatted as a displayed block + $self->{options}{'_default_translated'} .= " <label>"; + $self->{options}{'_default_break'} .= " <label>"; + + # legalnotice; contains text; Formatted as a displayed block + $self->{options}{'_default_translated'} .= " <legalnotice>"; + $self->{options}{'_default_break'} .= " <legalnotice>"; + + # lhs; contains text; Formatted as a displayed block. + # NOTE: it might be better to have the production as verbatim + # Keeping the constrainst inline to have it close to the + # lhs or rhs. + $self->{options}{'_default_translated'} .= " <lhs>"; + $self->{options}{'_default_break'} .= " <lhs>"; + + # lineage; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <lineage>"; + $self->{options}{'_default_inline'} .= " <lineage>"; + + # lineannotation; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <lineannotation>"; + $self->{options}{'_default_inline'} .= " <lineannotation>"; + + # link; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <link>"; + $self->{options}{'_default_inline'} .= " <link>"; + + # listitem; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <listitem>"; + $self->{options}{'_default_break'} .= " <listitem>"; + + # literal; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <literal>"; + $self->{options}{'_default_inline'} .= " <literal>"; + + # literallayout; contains text; verbatim + $self->{options}{'_default_translated'} .= " W<literallayout>"; + $self->{options}{'_default_placeholder'} .= " <literallayout>"; + + # locator; does not contain text; + $self->{options}{'_default_untranslated'} .= " <locator>"; + $self->{options}{'_default_inline'} .= " <locator>"; + + # lot; does not contain text; Formatted as a displayed block. + # v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <lot>"; + $self->{options}{'_default_break'} .= " <lot>"; + + # lotentry; contains text; Formatted as a displayed block. + # v4, not in v5 + $self->{options}{'_default_translated'} .= " <lotentry>"; + $self->{options}{'_default_break'} .= " <lotentry>"; + +# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM + + # manvolnum; contains text; + $self->{options}{'_default_translated'} .= " <manvolnum>"; + $self->{options}{'_default_inline'} .= " <manvolnum>"; + + # markup; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <markup>"; + $self->{options}{'_default_inline'} .= " <markup>"; + + # mathphrase; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <mathphrase>"; + $self->{options}{'_default_inline'} .= " <mathphrase>"; + + # medialabel; contains text; Formatted inline + # v4, not in v5 + $self->{options}{'_default_translated'} .= " <medialabel>"; + $self->{options}{'_default_inline'} .= " <medialabel>"; + + # mediaobject; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <mediaobject>"; + $self->{options}{'_default_placeholder'} .= " <mediaobject>"; + + # mediaobjectco; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <mediaobjectco>"; + $self->{options}{'_default_placeholder'} .= " <mediaobjectco>"; + + # member; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <member>"; + $self->{options}{'_default_inline'} .= " <member>"; + + # menuchoice; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <menuchoice>"; + $self->{options}{'_default_inline'} .= " <menuchoice>"; + + # methodname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <methodname>"; + $self->{options}{'_default_inline'} .= " <methodname>"; + + # methodparam; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <methodparam>"; + $self->{options}{'_default_inline'} .= " <methodparam>"; + + # methodsynopsis; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <methodsynopsis>"; + $self->{options}{'_default_inline'} .= " <methodsynopsis>"; + + # modifier; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <modifier>"; + $self->{options}{'_default_inline'} .= " <modifier>"; + + # mousebutton; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <mousebutton>"; + $self->{options}{'_default_inline'} .= " <mousebutton>"; + + # msg; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msg>"; + $self->{options}{'_default_break'} .= " <msg>"; + + # msgaud; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <msgaud>"; + $self->{options}{'_default_break'} .= " <msgaud>"; + + # msgentry; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgentry>"; + $self->{options}{'_default_break'} .= " <msgentry>"; + + # msgexplan; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgexplan>"; + $self->{options}{'_default_break'} .= " <msgexplan>"; + + # msginfo; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msginfo>"; + $self->{options}{'_default_break'} .= " <msginfo>"; + + # msglevel; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <msglevel>"; + $self->{options}{'_default_break'} .= " <msglevel>"; + + # msgmain; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgmain>"; + $self->{options}{'_default_break'} .= " <msgmain>"; + + # msgorig; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <msgorig>"; + $self->{options}{'_default_break'} .= " <msgorig>"; + + # msgrel; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgrel>"; + $self->{options}{'_default_break'} .= " <msgrel>"; + + # msgset; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgset>"; + $self->{options}{'_default_placeholder'} .= " <msgset>"; + + # msgsub; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgsub>"; + $self->{options}{'_default_break'} .= " <msgsub>"; + + # msgtext; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <msgtext>"; + $self->{options}{'_default_break'} .= " <msgtext>"; + +# NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN + + # nonterminal; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <nonterminal>"; + $self->{options}{'_default_inline'} .= " <nonterminal>"; + + # note; does not contain text; Formatted inline + # NOTE: can be in a para + $self->{options}{'_default_untranslated'} .= " <note>"; + $self->{options}{'_default_inline'} .= " <note>"; + +# OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO + + # objectinfo; does not contain text; v3.1 -> v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <objectinfo>"; + $self->{options}{'_default_placeholder'} .= " <objectinfo>"; + + # olink; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <olink>"; + $self->{options}{'_default_inline'} .= " <olink>"; + + # ooclass; does not contain text; Formatted inline + $self->{options}{'_default_translated'} .= " <ooclass>"; + $self->{options}{'_default_inline'} .= " <ooclass>"; + + # ooexception; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <ooexception>"; + $self->{options}{'_default_inline'} .= " <ooexception>"; + + # oointerface; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <oointerface>"; + $self->{options}{'_default_inline'} .= " <oointerface>"; + + # option; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <option>"; + $self->{options}{'_default_inline'} .= " <option>"; + + # optional; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <optional>"; + $self->{options}{'_default_inline'} .= " <optional>"; + + # orderedlist; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <orderedlist>"; + $self->{options}{'_default_placeholder'} .= " <orderedlist>"; + + # org; does not contain text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_untranslated'} .= " <org>"; + $self->{options}{'_default_inline'} .= " <org>"; + + # orgdiv; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <orgdiv>"; + $self->{options}{'_default_inline'} .= " <orgdiv>"; + + # orgname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <orgname>"; + $self->{options}{'_default_inline'} .= " <orgname>"; + + # otheraddr; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <otheraddr>"; + $self->{options}{'_default_inline'} .= " <otheraddr>"; + + # othercredit; does not contain text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_untranslated'} .= " <othercredit>"; + $self->{options}{'_default_inline'} .= " <othercredit>"; + + # othername; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <othername>"; + $self->{options}{'_default_inline'} .= " <othername>"; + +# PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP + + # package; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <package>"; + $self->{options}{'_default_inline'} .= " <package>"; + + # pagenums; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <pagenums>"; + $self->{options}{'_default_inline'} .= " <pagenums>"; + + # para; contains text; Formatted as a displayed block + $self->{options}{'_default_translated'} .= " <para>"; + $self->{options}{'_default_break'} .= " <para>"; + + # paramdef; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <paramdef>"; + $self->{options}{'_default_inline'} .= " <paramdef>"; + + # parameter; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <parameter>"; + $self->{options}{'_default_inline'} .= " <parameter>"; + + # part; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <part>"; + $self->{options}{'_default_break'} .= " <part>"; + + # partinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <partinfo>"; + $self->{options}{'_default_placeholder'} .= " <partinfo>"; + + # partintro; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <partintro>"; + $self->{options}{'_default_break'} .= " <partintro>"; + + # person; does not contain text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_untranslated'} .= " <person>"; + $self->{options}{'_default_inline'} .= " <person>"; + + # personblurb; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <personblurb>"; + $self->{options}{'_default_placeholder'} .= " <personblurb>"; + + # personname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <personname>"; + $self->{options}{'_default_inline'} .= " <personname>"; + + # phone; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <phone>"; + $self->{options}{'_default_inline'} .= " <phone>"; + + # phrase; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <phrase>"; + $self->{options}{'_default_inline'} .= " <phrase>"; + + # pob; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <pob>"; + $self->{options}{'_default_inline'} .= " <pob>"; + + # postcode; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <postcode>"; + $self->{options}{'_default_inline'} .= " <postcode>"; + + # preface; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <preface>"; + $self->{options}{'_default_break'} .= " <preface>"; + + # prefaceinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <prefaceinfo>"; + $self->{options}{'_default_placeholder'} .= " <prefaceinfo>"; + + # primary; contains text; + $self->{options}{'_default_translated'} .= " <primary>"; + $self->{options}{'_default_break'} .= " <primary>"; + + # primaryie; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <primaryie>"; + $self->{options}{'_default_break'} .= " <primaryie>"; + + # printhistory; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <printhistory>"; + $self->{options}{'_default_break'} .= " <printhistory>"; + + # procedure; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <procedure>"; + $self->{options}{'_default_placeholder'} .= " <procedure>"; + + # production; doesnot contain text; + # NOTE: it might be better to have the production as verbatim + # Keeping the constrainst inline to have it close to the + # lhs or rhs. + $self->{options}{'_default_untranslated'} .= " <production>"; + $self->{options}{'_default_break'} .= " <production>"; + + # productionrecap; does not contain text; like production + $self->{options}{'_default_untranslated'} .= " <productionrecap>"; + $self->{options}{'_default_break'} .= " <productionrecap>"; + + # productionset; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <productionset>"; + $self->{options}{'_default_placeholder'} .= " <productionset>"; + + # productname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <productname>"; + $self->{options}{'_default_inline'} .= " <productname>"; + + # productnumber; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <productnumber>"; + $self->{options}{'_default_inline'} .= " <productnumber>"; + + # programlisting; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " W<programlisting>"; + $self->{options}{'_default_placeholder'} .= " <programlisting>"; + + # programlistingco; contains text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <programlistingco>"; + $self->{options}{'_default_placeholder'} .= " <programlistingco>"; + + # prompt; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <prompt>"; + $self->{options}{'_default_inline'} .= " <prompt>"; + + # property; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <property>"; + $self->{options}{'_default_inline'} .= " <property>"; + + # pubdate; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <pubdate>"; + $self->{options}{'_default_inline'} .= " <pubdate>"; + + # publisher; does not contain text; Formatted inline or as a displayed block + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <publisher>"; + $self->{options}{'_default_inline'} .= " <publisher>"; + + # publishername; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_translated'} .= " <publishername>"; + $self->{options}{'_default_inline'} .= " <publishername>"; + +# QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ + + # qandadiv; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <qandadiv>"; + $self->{options}{'_default_break'} .= " <qandadiv>"; + + # qandaentry; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <qandaentry>"; + $self->{options}{'_default_break'} .= " <qandaentry>"; + + # qandaset; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <qandaset>"; + $self->{options}{'_default_break'} .= " <qandaset>"; + + # question; does not contain text; + $self->{options}{'_default_untranslated'} .= " <question>"; + $self->{options}{'_default_break'} .= " <question>"; + + # quote; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <quote>"; + $self->{options}{'_default_inline'} .= " <quote>"; + +# RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR + + # refclass; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <refclass>"; + $self->{options}{'_default_break'} .= " <refclass>"; + + # refdescriptor; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <refdescriptor>"; + $self->{options}{'_default_break'} .= " <refdescriptor>"; + + # refentry; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refentry>"; + $self->{options}{'_default_break'} .= " <refentry>"; + + # refentryinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <refentryinfo>"; + $self->{options}{'_default_placeholder'} .= " <refentryinfo>"; + + # refentrytitle; contains text; Formatted as a displayed block +# FIXME: do not seems to be a block + $self->{options}{'_default_translated'} .= " <refentrytitle>"; + $self->{options}{'_default_inline'} .= " <refentrytitle>"; + + # reference; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <reference>"; + $self->{options}{'_default_break'} .= " <reference>"; + + # referenceinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <referenceinfo>"; + $self->{options}{'_default_placeholder'} .= " <referenceinfo>"; + + # refmeta; does not contains text; + # NOTE: could be in the inline class + $self->{options}{'_default_untranslated'} .= " <refmeta>"; + $self->{options}{'_default_break'} .= " <refmeta>"; + + # refmiscinfo; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <refmiscinfo>"; + $self->{options}{'_default_break'} .= " <refmiscinfo>"; + + # refname; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <refname>"; + $self->{options}{'_default_break'} .= " <refname>"; + + # refnamediv; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refnamediv>"; + $self->{options}{'_default_break'} .= " <refnamediv>"; + + # refpurpose; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <refpurpose>"; + $self->{options}{'_default_inline'} .= " <refpurpose>"; + + # refsect1; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refsect1>"; + $self->{options}{'_default_break'} .= " <refsect1>"; + + # refsect1info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <refsect1info>"; + $self->{options}{'_default_placeholder'} .= " <refsect1info>"; + + # refsect2; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refsect2>"; + $self->{options}{'_default_break'} .= " <refsect2>"; + + # refsect2info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <refsect2info>"; + $self->{options}{'_default_placeholder'} .= " <refsect2info>"; + + # refsect3; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refsect3>"; + $self->{options}{'_default_break'} .= " <refsect3>"; + + # refsect3info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <refsect3info>"; + $self->{options}{'_default_placeholder'} .= " <refsect3info>"; + + # refsection; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refsection>"; + $self->{options}{'_default_break'} .= " <refsection>"; + + # refsectioninfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <refsectioninfo>"; + $self->{options}{'_default_placeholder'} .= " <refsectioninfo>"; + + # refsynopsisdiv; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <refsynopsisdiv>"; + $self->{options}{'_default_break'} .= " <refsynopsisdiv>"; + + # refsynopsisdivinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <refsynopsisdivinfo>"; + $self->{options}{'_default_placeholder'} .= " <refsynopsisdivinfo>"; + + # releaseinfo; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <releaseinfo>"; + $self->{options}{'_default_break'} .= " <releaseinfo>"; + + # remark; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_translated'} .= " <remark>"; + $self->{options}{'_default_inline'} .= " <remark>"; + + # replaceable; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <replaceable>"; + $self->{options}{'_default_inline'} .= " <replaceable>"; + + # returnvalue; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <returnvalue>"; + $self->{options}{'_default_inline'} .= " <returnvalue>"; + + # revdescription; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_translated'} .= " <revdescription>"; + $self->{options}{'_default_break'} .= " <revdescription>"; + + # revhistory; does not contain text; Formatted as a displayed block + $self->{options}{'_default_untranslated'} .= " <revhistory>"; + $self->{options}{'_default_break'} .= " <revhistory>"; + + # revision; does not contain text; + $self->{options}{'_default_untranslated'} .= " <revision>"; + $self->{options}{'_default_break'} .= " <revision>"; + + # revnumber; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <revnumber>"; + $self->{options}{'_default_inline'} .= " <revnumber>"; + + # revremark; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_translated'} .= " <revremark>"; + $self->{options}{'_default_break'} .= " <revremark>"; + + # rhs; contains text; Formatted as a displayed block. + # NOTE: it might be better to have the production as verbatim + # Keeping the constrainst inline to have it close to the + # lhs or rhs. + $self->{options}{'_default_translated'} .= " <rhs>"; + $self->{options}{'_default_break'} .= " <rhs>"; + + # row; does not contain text; + $self->{options}{'_default_untranslated'} .= " <row>"; + $self->{options}{'_default_break'} .= " <row>"; + +# SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS + + # sbr; does not contain text; line break + $self->{options}{'_default_untranslated'} .= " <sbr>"; + $self->{options}{'_default_break'} .= " <sbr>"; + + # screen; contains text; verbatim + $self->{options}{'_default_translated'} .= " W<screen>"; + $self->{options}{'_default_placeholder'} .= " <screen>"; + + # screenco; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <screenco>"; + $self->{options}{'_default_placeholder'} .= " <screenco>"; + + # screeninfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <screeninfo>"; + $self->{options}{'_default_placeholder'} .= " <screeninfo>"; + + # screenshot; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <screenshot>"; + $self->{options}{'_default_placeholder'} .= " <screenshot>"; + + # secondary; contains text; + $self->{options}{'_default_translated'} .= " <secondary>"; + $self->{options}{'_default_break'} .= " <secondary>"; + + # secondaryie; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <secondaryie>"; + $self->{options}{'_default_break'} .= " <secondaryie>"; + + # sect1; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <sect1>"; + $self->{options}{'_default_break'} .= " <sect1>"; + + # sect1info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sect1info>"; + $self->{options}{'_default_placeholder'} .= " <sect1info>"; + + # sect2; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <sect2>"; + $self->{options}{'_default_break'} .= " <sect2>"; + + # sect2info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sect2info>"; + $self->{options}{'_default_placeholder'} .= " <sect2info>"; + + # sect3; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <sect3>"; + $self->{options}{'_default_break'} .= " <sect3>"; + + # sect3info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sect3info>"; + $self->{options}{'_default_placeholder'} .= " <sect3info>"; + + # sect4; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <sect4>"; + $self->{options}{'_default_break'} .= " <sect4>"; + + # sect4info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sect4info>"; + $self->{options}{'_default_placeholder'} .= " <sect4info>"; + + # sect5; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <sect5>"; + $self->{options}{'_default_break'} .= " <sect5>"; + + # sect5info; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sect5info>"; + $self->{options}{'_default_placeholder'} .= " <sect5info>"; + + # section; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <section>"; + $self->{options}{'_default_break'} .= " <section>"; + + # sectioninfo; does not contain text; v3.1 -> v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sectioninfo>"; + $self->{options}{'_default_placeholder'} .= " <sectioninfo>"; + + # see; contains text; + $self->{options}{'_default_translated'} .= " <see>"; + $self->{options}{'_default_break'} .= " <see>"; + + # seealso; contains text; + $self->{options}{'_default_translated'} .= " <seealso>"; + $self->{options}{'_default_break'} .= " <seealso>"; + + # seealsoie; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <seealsoie>"; + $self->{options}{'_default_break'} .= " <seealsoie>"; + + # seeie; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <seeie>"; + $self->{options}{'_default_break'} .= " <seeie>"; + + # seg; contains text; + $self->{options}{'_default_translated'} .= " <seg>"; + $self->{options}{'_default_break'} .= " <seg>"; + + # seglistitem; does not contain text; + $self->{options}{'_default_untranslated'} .= " <seglistitem>"; + $self->{options}{'_default_break'} .= " <seglistitem>"; + + # segmentedlist; does not contain text; + $self->{options}{'_default_untranslated'} .= " <segmentedlist>"; + $self->{options}{'_default_break'} .= " <segmentedlist>"; + + # segtitle; contains text; + $self->{options}{'_default_translated'} .= " <segtitle>"; + $self->{options}{'_default_break'} .= " <segtitle>"; + + # seriesinfo; does not contain text; + # Removed in v4.0 + $self->{options}{'_default_untranslated'} .= " <seriesinfo>"; + $self->{options}{'_default_placeholder'} .= " <seriesinfo>"; + + # seriesvolnums; contains text; Formatted inline + # NOTE: could be in the break class + $self->{options}{'_default_translated'} .= " <seriesvolnums>"; + $self->{options}{'_default_inline'} .= " <seriesvolnums>"; + + # set; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <set>"; + $self->{options}{'_default_break'} .= " <set>"; + + # setindex; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <setindex>"; + $self->{options}{'_default_break'} .= " <setindex>"; + + # setindexinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <setindexinfo>"; + $self->{options}{'_default_placeholder'} .= " <setindexinfo>"; + + # setinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <setinfo>"; + $self->{options}{'_default_placeholder'} .= " <setinfo>"; + + # sgmltag; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <sgmltag>"; + $self->{options}{'_default_inline'} .= " <sgmltag>"; + + # shortaffil; contains text; Formatted inline or as a + # displayed block depending on context + $self->{options}{'_default_translated'} .= " <shortaffil>"; + $self->{options}{'_default_inline'} .= " <shortaffil>"; + + # shortcut; does not contain text; Formatted inline + $self->{options}{'_default_untranslated'} .= " <shortcut>"; + $self->{options}{'_default_inline'} .= " <shortcut>"; + + # sidebar; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <sidebar>"; + $self->{options}{'_default_break'} .= " <sidebar>"; + + # sidebarinfo; does not contain text; v4, not in v5 + $self->{options}{'_default_untranslated'} .= " <sidebarinfo>"; + $self->{options}{'_default_placeholder'} .= " <sidebarinfo>"; + + # simpara; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <simpara>"; + $self->{options}{'_default_break'} .= " <simpara>"; + + # simplelist; does not contain text; + $self->{options}{'_default_untranslated'} .= " <simplelist>"; + $self->{options}{'_default_inline'} .= " <simplelist>"; + + # simplemsgentry; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <simplemsgentry>"; + $self->{options}{'_default_break'} .= " <simplemsgentry>"; + + # simplesect; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <simplesect>"; + $self->{options}{'_default_break'} .= " <simplesect>"; + + # spanspec; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <spanspec>"; + $self->{options}{'_default_break'} .= " <spanspec>"; + + # state; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <state>"; + $self->{options}{'_default_inline'} .= " <state>"; + + # step; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <step>"; + $self->{options}{'_default_break'} .= " <step>"; + + # stepalternatives; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <stepalternatives>"; + $self->{options}{'_default_break'} .= " <stepalternatives>"; + + # street; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <street>"; + $self->{options}{'_default_inline'} .= " <street>"; + + # structfield; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <structfield>"; + $self->{options}{'_default_inline'} .= " <structfield>"; + + # structname; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <structname>"; + $self->{options}{'_default_inline'} .= " <structname>"; + + # subject; does not contain text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_untranslated'} .= " <subject>"; + $self->{options}{'_default_break'} .= " <subject>"; + + # subjectset; does not contain text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_untranslated'} .= " <subjectset>"; + $self->{options}{'_default_break'} .= " <subjectset>"; + + # subjectterm; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <subjectterm>"; + $self->{options}{'_default_break'} .= " <subjectterm>"; + + # subscript; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <subscript>"; + $self->{options}{'_default_inline'} .= " <subscript>"; + + # substeps; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <substeps>"; + $self->{options}{'_default_break'} .= " <substeps>"; + + # subtitle; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <subtitle>"; + $self->{options}{'_default_break'} .= " <subtitle>"; + + # superscript; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <superscript>"; + $self->{options}{'_default_inline'} .= " <superscript>"; + + # surname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <surname>"; + $self->{options}{'_default_inline'} .= " <surname>"; + +#svg:svg + + # symbol; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <symbol>"; + $self->{options}{'_default_inline'} .= " <symbol>"; + + # synopfragment; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <synopfragment>"; + $self->{options}{'_default_placeholder'} .= " <synopfragment>"; + + # synopfragmentref; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <synopfragmentref>"; + $self->{options}{'_default_inline'} .= " <synopfragmentref>"; + + # synopsis; contains text; verbatim + $self->{options}{'_default_translated'} .= " W<synopsis>"; + $self->{options}{'_default_placeholder'} .= " <synopsis>"; + + # systemitem; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <systemitem>"; + $self->{options}{'_default_inline'} .= " <systemitem>"; + +# TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT + + # table; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <table>"; + $self->{options}{'_default_placeholder'} .= " <table>"; + + # tag; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <tag>"; + $self->{options}{'_default_inline'} .= " <tag>"; + + # task; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <task>"; + $self->{options}{'_default_placeholder'} .= " <task>"; + + # taskprerequisites; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <taskprerequisites>"; + $self->{options}{'_default_break'} .= " <taskprerequisites>"; + + # taskrelated; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <taskrelated>"; + $self->{options}{'_default_break'} .= " <taskrelated>"; + + # tasksummary; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <tasksummary>"; + $self->{options}{'_default_break'} .= " <tasksummary>"; + + # tbody; does not contain text; + $self->{options}{'_default_untranslated'} .= " <tbody>"; + $self->{options}{'_default_break'} .= " <tbody>"; + + # td; contains text; + $self->{options}{'_default_translated'} .= " <td>"; + $self->{options}{'_default_break'} .= " <td>"; + + # term; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <term>"; + $self->{options}{'_default_break'} .= " <term>"; + + # termdef; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <termdef>"; + $self->{options}{'_default_inline'} .= " <termdef>"; + + # tertiary; contains text; Suppressed + $self->{options}{'_default_translated'} .= " <tertiary>"; + $self->{options}{'_default_placeholder'} .= " <tertiary>"; + + # tertiaryie; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <tertiaryie>"; + $self->{options}{'_default_break'} .= " <tertiaryie>"; + + # textdata; does not contain text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_untranslated'} .= " <textdata>"; + $self->{options}{'_default_break'} .= " <textdata>"; + $self->{options}{'_default_attributes'}.=' <textdata>fileref'; + + # textobject; does not contain text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_untranslated'} .= " <textobject>"; + $self->{options}{'_default_break'} .= " <textobject>"; + + # tfoot; does not contain text; + $self->{options}{'_default_untranslated'} .= " <tfoot>"; + $self->{options}{'_default_break'} .= " <tfoot>"; + + # tgroup; does not contain text; + $self->{options}{'_default_untranslated'} .= " <tgroup>"; + $self->{options}{'_default_break'} .= " <tgroup>"; + + # th; contains text; + $self->{options}{'_default_translated'} .= " <th>"; + $self->{options}{'_default_break'} .= " <th>"; + + # thead; does not contain text; + $self->{options}{'_default_untranslated'} .= " <thead>"; + $self->{options}{'_default_break'} .= " <thead>"; + + # tip; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <tip>"; + $self->{options}{'_default_break'} .= " <tip>"; + + # title; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <title>"; + $self->{options}{'_default_break'} .= " <title>"; + + # titleabbrev; contains text; Formatted inline or as a displayed block + # NOTE: could be in the inline class + $self->{options}{'_default_translated'} .= " <titleabbrev>"; + $self->{options}{'_default_break'} .= " <titleabbrev>"; + + # toc; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <toc>"; + $self->{options}{'_default_break'} .= " <toc>"; + + # tocback; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <tocback>"; + $self->{options}{'_default_break'} .= " <tocback>"; + + # tocchap; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <tocchap>"; + $self->{options}{'_default_break'} .= " <tocchap>"; + + # tocdiv; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <tocdiv>"; + $self->{options}{'_default_break'} .= " <tocdiv>"; + + # tocentry; contains text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <tocentry>"; + $self->{options}{'_default_break'} .= " <tocentry>"; + + # tocfront; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_translated'} .= " <tocfront>"; + $self->{options}{'_default_break'} .= " <tocfront>"; + + # toclevel1; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <toclevel1>"; + $self->{options}{'_default_break'} .= " <toclevel1>"; + + # toclevel2; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <toclevel2>"; + $self->{options}{'_default_break'} .= " <toclevel2>"; + + # toclevel3; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <toclevel3>"; + $self->{options}{'_default_break'} .= " <toclevel3>"; + + # toclevel4; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <toclevel4>"; + $self->{options}{'_default_break'} .= " <toclevel4>"; + + # toclevel5; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <toclevel5>"; + $self->{options}{'_default_break'} .= " <toclevel5>"; + + # tocpart; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <tocpart>"; + $self->{options}{'_default_break'} .= " <tocpart>"; + + # token; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <token>"; + $self->{options}{'_default_inline'} .= " <token>"; + + # tr; does not contain text; + $self->{options}{'_default_untranslated'} .= " <tr>"; + $self->{options}{'_default_break'} .= " <tr>"; + + # trademark; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <trademark>"; + $self->{options}{'_default_inline'} .= " <trademark>"; + + # type; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <type>"; + $self->{options}{'_default_inline'} .= " <type>"; + +# UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU + + # ulink; contains text; Formatted inline; v4, not in v5 + $self->{options}{'_default_translated'} .= " <ulink>"; + $self->{options}{'_default_inline'} .= " <ulink>"; + + # uri; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <uri>"; + $self->{options}{'_default_inline'} .= " <uri>"; + + # userinput; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <userinput>"; + $self->{options}{'_default_inline'} .= " <userinput>"; + +# VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV + + # varargs; empty element; + $self->{options}{'_default_untranslated'} .= " <varargs>"; + $self->{options}{'_default_inline'} .= " <varargs>"; + + # variablelist; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <variablelist>"; + $self->{options}{'_default_placeholder'} .= " <variablelist>"; + + # varlistentry; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <varlistentry>"; + $self->{options}{'_default_break'} .= " <varlistentry>"; + + # varname; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <varname>"; + $self->{options}{'_default_inline'} .= " <varname>"; + + # videodata; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_untranslated'} .= " <videodata>"; + $self->{options}{'_default_break'} .= " <videodata>"; + $self->{options}{'_default_attributes'}.=' <videodata>fileref'; + + # videoobject; contains text; Formatted inline or as a displayed block + $self->{options}{'_default_untranslated'} .= " <videoobject>"; + $self->{options}{'_default_break'} .= " <videoobject>"; + + # void; empty element; + $self->{options}{'_default_untranslated'} .= " <void>"; + $self->{options}{'_default_inline'} .= " <void>"; + + # volumenum; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <volumenum>"; + $self->{options}{'_default_inline'} .= " <volumenum>"; + +# WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW + + # warning; does not contain text; Formatted as a displayed block. + $self->{options}{'_default_untranslated'} .= " <warning>"; + $self->{options}{'_default_break'} .= " <warning>"; + + # wordasword; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <wordasword>"; + $self->{options}{'_default_inline'} .= " <wordasword>"; + +# XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX + + # xref; empty element; + $self->{options}{'_default_untranslated'} .= " <xref>"; + $self->{options}{'_default_inline'} .= " <xref>"; + +# YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY + + # year; contains text; Formatted inline + $self->{options}{'_default_translated'} .= " <year>"; + $self->{options}{'_default_inline'} .= " <year>"; + +# ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ + + $self->{options}{'_default_attributes'}.=' + lang + xml:lang'; + + $self->treat_options; +} Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Pod.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Pod.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Pod.pm (revision 2840) @@ -0,0 +1,323 @@ +# Locale::Po4a::Pod -- Convert POD data to PO file, for translation. +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING file). +# +# This module converts POD to PO file, so that it becomes possible to +# translate POD formatted documentation. See gettext documentation for +# more info about PO files. + +############################################################################ +# Modules and declarations +############################################################################ + +use Pod::Parser; +use Locale::Po4a::TransTractor qw(process new get_out_charset); + +package Locale::Po4a::Pod; + +use 5.006; +use strict; +use warnings; + +require Exporter; + +use vars qw(@ISA); +@ISA = qw(Locale::Po4a::TransTractor Pod::Parser); + +sub initialize {} + +sub translate { + my ($self,$str,$ref,$type) = @_; + my (%options)=@_; + + $str = $self->pre_trans($str,$ref,$type); + $str = $self->SUPER::translate($str, $ref, $type, %options); + $str = $self->post_trans($str,$ref,$type); + + return $str; +} + +sub pre_trans { + my ($self,$str,$ref,$type)=@_; + + return $str; +} + +sub post_trans { + my ($self,$str,$ref,$type)=@_; + + # Change ascii non-breaking space to POD one + my $nbs_out = "\xA0"; + my $enc_length = Encode::from_to($nbs_out, "latin1", + $self->get_out_charset); + if (defined $enc_length) { + while ($str =~ m/(^|.*\s)(\S+?)\Q$nbs_out\E(\S+?)(\s.*$|$)/s) { + my ($begin, $m1, $m2, $end) = ($1, $2, $3, $4); + $str = (defined $begin)?$begin:""; + # Remove the non-breaking spaces in the string that will be + # between S<...> + $m2 =~ s/\Q$nbs_out\E/ /g; + $str .= "S<$m1 $m2>"; + $str .= (defined $end)?$end:""; + } + } + + return $str; +} + + +sub command { + my ($self, $command, $paragraph, $line_num) = @_; +# print STDOUT "cmd: '$command' '$paragraph' at $line_num\n"; + if ($command eq 'back' + || $command eq 'cut' + || $command eq 'pod') { + $self->pushline("=$command\n\n"); + } elsif ($command eq 'over') { + $self->pushline("=$command $paragraph".(length($paragraph)?"":"\n\n")); + } elsif ($command eq 'encoding') { + my $charset = $paragraph; + $charset =~ s/^\s*(.*?)\s*$/$1/s; + $self->detected_charset($charset) + # The =encoding line will be added by docheader + } else { + $paragraph=$self->translate($paragraph, + $self->input_file().":$line_num", + "=$command", + "wrap"=>1); + $self->pushline("=$command $paragraph\n\n"); + } +} + +sub verbatim { + my ($self, $paragraph, $line_num) = @_; +# print "verb: '$paragraph' at $line_num\n"; + + if ($paragraph eq "\n") { + $self->pushline("$paragraph\n"); + return; + } + $paragraph=$self->translate($paragraph, + $self->input_file().":$line_num", + "verbatim"); + $paragraph =~ s/\n$//m; + $self->pushline("$paragraph\n"); +} + +sub textblock { + my ($self, $paragraph, $line_num) = @_; +# print "text: '$paragraph' at $line_num\n"; + + # Fix a pretty damned bug. + # Podlators don't wrap explicitelly the text, and groff won't seem to + # wrap any line begining with a space. So, we have to consider as + # verbatim not only the paragraphs whose first line is indented, but + # the paragraph containing an indented line. + # That way, we'll declare more paragraphs as verbatim than needed, but + # that's harmless (only less confortable for translators). + + if ($paragraph eq "\n") { + $self->pushline("$paragraph\n"); + return; + } + if ($paragraph =~ m/^[ \t]/m) { + $self->verbatim($paragraph, $line_num) ; + return; + } + + $paragraph=$self->translate($paragraph, + $self->input_file().":$line_num", + 'textblock', + "wrap"=>1); + $paragraph=~ s/ +\n/\n/gm; + $self->pushline("$paragraph\n\n"); +} + +sub end_pod {} + +sub read { + my ($self,$filename)=@_; + + push @{$self->{DOCPOD}{infile}}, $filename; + $self->Locale::Po4a::TransTractor::read($filename); +} + +sub parse { + my $self=shift; + map {$self->parse_from_file($_)} @{$self->{DOCPOD}{infile}}; +} + +sub docheader { + my $self=shift; + my $encoding = $self->get_out_charset(); + if ( (defined $encoding) + and (length $encoding) + and ($encoding ne "ascii")) { + $encoding = "\n=encoding $encoding\n"; + } else { + $encoding = ""; + } + + return <<EOT; + + ***************************************************** + * GENERATED FILE, DO NOT EDIT * + * THIS IS NO SOURCE FILE, BUT RESULT OF COMPILATION * + ***************************************************** + +This file was generated by po4a(7). Do not store it (in VCS, for example), +but store the PO file used as source file by po4a-translate. + +In fact, consider this as a binary, and the PO file as a regular .c file: +If the PO get lost, keeping this translation up-to-date will be harder. +$encoding +EOT +} +1; + +############################################################################## +# Module return value and documentation +############################################################################## + +1; +__END__ + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Pod - convert POD data from/to PO files + +=head1 SYNOPSIS + + use Locale::Po4a::Pod; + my $parser = Locale::Po4a::Pod->new (sentence => 0, width => 78); + + # Read POD from STDIN and write to STDOUT. + $parser->parse_from_filehandle; + + # Read POD from file.pod and write to file.txt. + $parser->parse_from_file ('file.pod', 'file.txt'); + +=head1 DESCRIPTION + +Locale::Po4a::Pod is a module to help the translation of documentation in +the POD format (the preferred language for documenting Perl) into other +[human] languages. + +=head1 STATUS OF THIS MODULE + +I think that this module is rock stable, and there is only one known bug +with F</usr/lib/perl5/Tk/MainWindow.pod> (and some other +pages, see below) which contains: + + C<" #n"> + +Lack of luck, in the po4a version, this was split on the space by the +wrapping. As result, in the original version, the man page contains + + " #n" + +and mine contains + + "" #n"" + +which is logic since CE<lt>foobarE<gt> is rewritten "foobar". + +Complete list of pages having this problem on my box (from 564 pages; note +that it depends on the chosen wrapping column): +/usr/lib/perl5/Tk/MainWindow.pod +/usr/share/perl/5.8.0/overload.pod +/usr/share/perl/5.8.0/pod/perlapi.pod +/usr/share/perl/5.8.0/pod/perldelta.pod +/usr/share/perl/5.8.0/pod/perlfaq5.pod +/usr/share/perl/5.8.0/pod/perlpod.pod +/usr/share/perl/5.8.0/pod/perlre.pod +/usr/share/perl/5.8.0/pod/perlretut.pod + + + +=head1 INTERNALS + +As a derived class from Pod::Parser, Locale::Po4a::Pod supports the same +methods and interfaces. See L<Pod::Parser> for all the details; briefly, +one creates a new parser with C<< Locale::Po4a::Pod->new() >> and then +calls either parse_from_filehandle() or parse_from_file(). + +new() can take options, in the form of key/value pairs, that control the +behavior of the parser. The recognized options common to all Pod::Parser +children are: + +=over 4 + +=item B<alt> + +If set to a true value, selects an alternate output format that, among other +things, uses a different heading style and marks B<=item> entries with a +colon in the left margin. Defaults to false. + +=item B<code> + +If set to a true value, the non-POD parts of the input file will be included +in the output. Useful for viewing code documented with POD blocks with the +POD rendered and the code left intact. + +=item B<indent> + +The number of spaces to indent regular text, and the default indentation for +B<=over> blocks. Defaults to 4. + +=item B<loose> + +If set to a true value, a blank line is printed after a B<=head1> heading. +If set to false (the default), no blank line is printed after B<=head1>, +although one is still printed after B<=head2>. This is the default because +it's the expected formatting for manual pages; if you're formatting +arbitrary text documents, setting this to true may result in more pleasing +output. + +=item B<quotes> + +Sets the quote marks used to surround CE<lt>> text. If the value is a +single character, it is used as both the left and right quote; if it is two +characters, the first character is used as the left quote and the second as +the right quote; and if it is four characters, the first two are used as +the left quote and the second two as the right quote. + +This may also be set to the special value B<none>, in which case no quote +marks are added around CE<lt>> text. + +=item B<sentence> + +If set to a true value, Locale::Po4a::Pod will assume that each sentence +ends in two spaces, and will try to preserve that spacing. If set to +false, all consecutive whitespace in non-verbatim paragraphs is compressed +into a single space. Defaults to true. + +=item B<width> + +The column at which to wrap text on the right-hand side. Defaults to 76. + +=back + +=head1 SEE ALSO + +L<Pod::Parser>, +L<Locale::Po4a::Man(3pm)>, +L<Locale::Po4a::TransTractor(3pm)>, +L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Denis Barbier <barbier@linuxfr.org> + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Text.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Text.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Text.pm (revision 2840) @@ -0,0 +1,930 @@ +#!/usr/bin/perl -w + +# Po4a::Text.pm +# +# extract and translate translatable strings from a text documents +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Text - convert text documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Text is a module to help the translation of text documents into +other [human] languages. + +Paragraphs are split on empty lines (or lines containing only spaces or +tabulations). + +If a paragraph contains a line starting by a space (or tabulation), this +paragraph won't be rewrapped. + +=cut + +package Locale::Po4a::Text; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +These are this module's particular options: + +=over + +=item B<nobullets> + +Deactivate detection of bullets. + +By default, when a bullet is detected, the bullet paragraph is not considered +as a verbatim paragraph (with the no-wrap flag in the PO file), but the module +rewraps this paragraph in the generated PO file and in the translation. + +=cut + +my $bullets = 1; + +=item B<tabs=>I<mode> + +Specify how tabulations shall be handled. The I<mode> can be any of: + +=over + +=item B<split> + +Lines with tabulations introduce breaks in the current paragraph. + +=item B<verbatim> + +Paragraph containing tabulations will not be re-wrapped. + +=back + +By default, tabulations are considered as spaces. + +=cut + +my $tabs = ""; + +=item B<breaks=>I<regex> + +A regular expression matching lines which introduce breaks. +The regular expression will be anchored so that the whole line must match. + +=cut + +my $breaks; + +=item B<debianchangelog> + +Handle the header and footer of +released versions, which only contain non translatable informations. + +=cut + +my $debianchangelog = 0; + +=item B<fortunes> + +Handle the fortunes format, which separate fortunes with a line which +consists in '%' or '%%', and use '%%' as the beginning of a comment. + +=cut + +my $fortunes = 0; + +=item B<markdown> + +Handle some special markup in Markdown-formatted texts. + +=cut + +my $markdown = 0; + +=item B<asciidoc> + +Handle documents in the AsciiDoc format. + +=cut + +my $asciidoc = 0; + +=item B<control>[B<=>I<taglist>] + +Handle control files. +A comma-separated list of tags to be translated can be provided. + +=cut + +my %control = (); + +my $parse_func = \&parse_fallback; + +my @comments = (); + +=back + +=cut + +sub initialize { + my $self = shift; + my %options = @_; + + $self->{options}{'control'} = ""; + $self->{options}{'asciidoc'} = 1; + $self->{options}{'breaks'} = 1; + $self->{options}{'debianchangelog'} = 1; + $self->{options}{'debug'} = 1; + $self->{options}{'fortunes'} = 1; + $self->{options}{'markdown'} = 1; + $self->{options}{'nobullets'} = 1; + $self->{options}{'tabs'} = 1; + $self->{options}{'verbose'} = 1; + + foreach my $opt (keys %options) { + die wrap_mod("po4a::text", + dgettext("po4a", "Unknown option: %s"), $opt) + unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options{$opt}; + } + + if (defined $options{'nobullets'}) { + $bullets = 0; + } + + if (defined $options{'tabs'}) { + $tabs = $options{'tabs'}; + } + + if (defined $options{'breaks'}) { + $breaks = $options{'breaks'}; + } + + if (defined $options{'debianchangelog'}) { + $parse_func = \&parse_debianchangelog; + } + + if (defined $options{'fortunes'}) { + $parse_func = \&parse_fortunes; + } + + if (defined $options{'markdown'}) { + $parse_func = \&parse_markdown; + $markdown=1; + } + + if (defined $options{'asciidoc'}) { + $parse_func = \&parse_asciidoc; + $asciidoc=1; + } + + if (defined $options{'control'}) { + $parse_func = \&parse_control; + if ($options{'control'} eq "1") { + $control{''}=1; + } else { + foreach my $tag (split(',',$options{'control'})) { + $control{$tag}=1; + } + } + } +} + +sub parse_fallback { + my ($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = @_; + if ( ($line =~ /^\s*$/) + or ( defined $breaks + and $line =~ m/^$breaks$/)) { + # Break paragraphs on lines containing only spaces + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1 unless defined($self->{verbatim}); + $self->pushline($line."\n"); + undef $self->{controlkey}; + } elsif ($line =~ /^-- $/) { + # Break paragraphs on email signature hint + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $self->pushline($line."\n"); + } elsif ( $line =~ /^=+$/ + or $line =~ /^_+$/ + or $line =~ /^-+$/) { + $wrapped_mode = 0; + $paragraph .= $line."\n"; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + } elsif ($tabs eq "split" and $line =~ m/\t/ and $paragraph !~ m/\t/s) { + $wrapped_mode = 0; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph = "$line\n"; + $wrapped_mode = 0; + } elsif ($tabs eq "split" and $line !~ m/\t/ and $paragraph =~ m/\t/s) { + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph = "$line\n"; + $wrapped_mode = 1; + } else { + if ($line =~ /^\s/) { + # A line starting by a space indicates a non-wrap + # paragraph + $wrapped_mode = 0; + } + if ($markdown and + ( $line =~ /\S $/ # explicit newline + or $line =~ /"""$/)) { # """ textblock inside macro begin + # Markdown markup needing separation _after_ this line + $end_of_paragraph = 1; + } else { + undef $self->{bullet}; + undef $self->{indent}; + } +# TODO: comments + $paragraph .= $line."\n"; + } + return ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); +} + +sub parse_debianchangelog { + my ($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = @_; + if ($expect_header and + $line =~ /^(\w[-+0-9a-z.]*)\ \(([^\(\) \t]+)\) # src, version + \s+([-+0-9a-z.]+); # distribution + \s*urgency\s*\=\s*(.*\S)\s*$/ix) { # + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $self->pushline("$line\n"); + $expect_header=0; + } elsif ($line =~ m/^ \-\- (.*) <(.*)> ((\w+\,\s*)?\d{1,2}\s+\w+\s+\d{4}\s+\d{1,2}:\d\d:\d\d\s+[-+]\d{4}(\s+\([^\\\(\)]\))?)$/) { + # Found trailer + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $self->pushline("$line\n"); + $expect_header=1; + } else { + return parse_fallback($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); + } + return ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); +} + +sub parse_fortunes { + my ($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = @_; + if ($line =~ m/^%%?\s*$/) { + # Found end of fortune + do_paragraph($self,$paragraph,$wrapped_mode); + $self->pushline("\n") unless ( $wrapped_mode == 0 + or $paragraph eq ""); + $paragraph=""; + $wrapped_mode = 1; + $self->pushline("$line\n"); + } else { + $line =~ s/%%(.*)$//; + } + return ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); +} + +sub parse_control { + my ($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = @_; + if ($line =~ m/^([^ :]*): *(.*)$/) { + warn "Unrecognized section: '$paragraph'\n" + unless $paragraph eq ""; + my $tag = $1; + my $val = $2; + my $t; + if ($control{''} or $control{$tag}) { + $t = $self->translate($val, + $self->{ref}, + $tag.(defined $self->{controlkey}?", ".$self->{controlkey}:""), + "wrap" => 0); + } else { + $t = $val; + } + if (not defined $self->{controlkey}) { + $self->{controlkey} = "$tag: $val"; + } + $self->pushline("$tag: $t\n"); + $paragraph=""; + $wrapped_mode = 1; + $self->{bullet} = ""; + $self->{indent} = " "; + } elsif ($line eq " .") { + do_paragraph($self,$paragraph,$wrapped_mode, + "Long Description".(defined $self->{controlkey}?", ".$self->{controlkey}:"")); + $paragraph=""; + $self->pushline($line."\n"); + $self->{bullet} = ""; + $self->{indent} = " "; + } elsif ($line =~ m/^ Link: +(.*)$/) { + do_paragraph($self,$paragraph,$wrapped_mode, + "Long Description".(defined $self->{controlkey}?", ".$self->{controlkey}:"")); + my $link=$1; + my $t1 = $self->translate("Link: ", + $self->{ref}, + "Link", + "wrap" => 0); + my $t2 = $self->translate($link, + $self->{ref}, + "Link".(defined $self->{controlkey}?", ".$self->{controlkey}:""), + "wrap" => 0); + $self->pushline(" $t1$t2\n"); + $paragraph=""; + } elsif (defined $self->{indent} and + $line =~ m/^$self->{indent}\S/) { + $paragraph .= $line."\n"; + $self->{type} = "Long Description".(defined $self->{controlkey}?", ".$self->{controlkey}:""); + } else { + return parse_fallback($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); + } + return ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); +} + +my $asciidoc_RE_SECTION_TEMPLATES = "sect1|sect2|sect3|sect4|preface|colophon|dedication|synopsis|index"; +my $asciidoc_RE_STYLE_ADMONITION = "TIP|NOTE|IMPORTANT|WARNING|CAUTION"; +my $asciidoc_RE_STYLE_PARAGRAPH = "normal|literal|verse|quote|listing|abstract|partintro|comment|example|sidebar|source|music|latex|graphviz"; +my $asciidoc_RE_STYLE_NUMBERING = "arabic|loweralpha|upperalpha|lowerroman|upperroman"; +my $asciidoc_RE_STYLE_LIST = "appendix|horizontal|qanda|glossary|bibliography"; +my $asciidoc_RE_STYLES = "$asciidoc_RE_SECTION_TEMPLATES|$asciidoc_RE_STYLE_ADMONITION|$asciidoc_RE_STYLE_PARAGRAPH|$asciidoc_RE_STYLE_NUMBERING|$asciidoc_RE_STYLE_LIST|float"; + +BEGIN { + my $UnicodeGCString_available = 0; + $UnicodeGCString_available = 1 if (eval { require Unicode::GCString }); + eval { + sub columns($$$) { + my $text = shift; + my $encoder = shift; + $text = $encoder->decode($text) if (defined($encoder) && $encoder->name ne "ascii"); + if ($UnicodeGCString_available) { + return Unicode::GCString->new($text)->columns(); + } else { + $text =~ s/\n$//s; + return length($text) if !(defined($encoder) && $encoder->name ne "ascii"); + die wrap_mod("po4a::text", + dgettext("po4a", "Detection of two line titles failed at %s\nInstall the Unicode::GCString module!"), shift) + } + } + }; +} + +sub parse_asciidoc { + my ($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = @_; + if ((defined $self->{verbatim}) and ($self->{verbatim} == 3)) { + # Untranslated blocks + $self->pushline($line."\n"); + if ($line =~ m/^~{4,}$/) { + undef $self->{verbatim}; + undef $self->{type}; + $wrapped_mode = 1; + } + } elsif ((defined $self->{verbatim}) and ($self->{verbatim} == 2)) { + # CommentBlock + if ($line =~ m/^\/{4,}$/) { + undef $self->{verbatim}; + undef $self->{type}; + $wrapped_mode = 1; + } else { + push @comments, $line; + } + } elsif ((not defined($self->{verbatim})) and ($line =~ m/^(\+|--)$/)) { + # List Item Continuation or List Block + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $self->pushline($line."\n"); + } elsif ((not defined($self->{verbatim})) and + ($line =~ m/^(={2,}|-{2,}|~{2,}|\^{2,}|\+{2,})$/) and + (defined($paragraph) )and + ($paragraph =~ m/^[^\n]*\n$/s) and + (columns($paragraph, $self->{TT}{po_in}{encoder}, $ref) == (length($line)))) { + # Found title + $wrapped_mode = 0; + my $level = $line; + $level =~ s/^(.).*$/$1/; + $paragraph =~ s/\n$//s; + my $t = $self->translate($paragraph, + $self->{ref}, + "Title $level", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline($t."\n"); + $paragraph=""; + @comments=(); + $wrapped_mode = 1; + $self->pushline(($level x (columns($t, $self->{TT}{po_in}{encoder}, $ref)))."\n"); + } elsif ($line =~ m/^(={1,5})( +)(.*?)( +\1)?$/) { + my $titlelevel1 = $1; + my $titlespaces = $2; + my $title = $3; + my $titlelevel2 = $4||""; + # Found one line title + do_paragraph($self,$paragraph,$wrapped_mode); + $wrapped_mode = 0; + $paragraph=""; + my $t = $self->translate($title, + $self->{ref}, + "Title $titlelevel1", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline($titlelevel1.$titlespaces.$t.$titlelevel2."\n"); + @comments=(); + $wrapped_mode = 1; + } elsif ($line =~ m/^(\/{4,}|\+{4,}|-{4,}|\.{4,}|\*{4,}|_{4,}|={4,}|~{4,}|\|={4,})$/) { + # Found one delimited block + my $t = $line; + $t =~ s/^(.).*$/$1/; + my $type = "delimited block $t"; + if (defined $self->{verbatim} and ($self->{type} ne $type)) { + $paragraph .= "$line\n"; + } else { + do_paragraph($self,$paragraph,$wrapped_mode); + if ( (defined $self->{type}) + and ($self->{type} eq $type)) { + undef $self->{type}; + undef $self->{verbatim}; + $wrapped_mode = 1; + } else { + if ($t eq "\/") { + # CommentBlock, should not be treated + $self->{verbatim} = 2; + } elsif ($t eq "+") { + # PassthroughBlock + $wrapped_mode = 0; + $self->{verbatim} = 1; + } elsif ($t eq "-" or $t eq "|") { + # ListingBlock + $wrapped_mode = 0; + $self->{verbatim} = 1; + } elsif ($t eq ".") { + # LiteralBlock + $wrapped_mode = 0; + $self->{verbatim} = 1; + } elsif ($t eq "*") { + # SidebarBlock + $wrapped_mode = 1; + } elsif ($t eq "_") { + # QuoteBlock + if ( (defined $self->{type}) + and ($self->{type} eq "verse")) { + $wrapped_mode = 0; + $self->{verbatim} = 1; + } else { + $wrapped_mode = 1; + } + } elsif ($t eq "=") { + # ExampleBlock + $wrapped_mode = 1; + } elsif ($t eq "~") { + # Filter blocks, TBC: not translated + $wrapped_mode = 0; + $self->{verbatim} = 3; + } + $self->{type} = $type; + } + $paragraph=""; + $self->pushline($line."\n") unless defined($self->{verbatim}) && $self->{verbatim} == 2; + } + } elsif ((not defined($self->{verbatim})) and ($line =~ m/^\/\/(.*)/)) { + # Comment line + push @comments, $1; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^\[\[([^\]]*)\]\]$/)) { + # Found BlockId + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $self->pushline($line."\n"); + undef $self->{bullet}; + undef $self->{indent}; + } elsif (not defined $self->{verbatim} and + ($paragraph eq "") and + ($line =~ m/^((?:$asciidoc_RE_STYLE_ADMONITION):\s+)(.*)$/)) { + my $type = $1; + my $text = $2; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=$text."\n"; + $wrapped_mode = 1; + $self->pushline($type); + undef $self->{bullet}; + undef $self->{indent}; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^\[($asciidoc_RE_STYLES)\]$/)) { + my $type = $1; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $self->pushline($line."\n"); + if ($type eq "verse") { + $wrapped_mode = 0; + } + undef $self->{bullet}; + undef $self->{indent}; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^\[(['"]?)(verse|quote)\1, +(.*)\]$/)) { + my $quote = $1 || ''; + my $type = $2; + my $arg = $3; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + my $t = $self->translate($arg, + $self->{ref}, + "$type", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline("[$quote$type$quote, $t]\n"); + @comments=(); + $wrapped_mode = 1; + if ($type eq "verse") { + $wrapped_mode = 0; + } + $self->{type} = $type; + undef $self->{bullet}; + undef $self->{indent}; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^\[icon="(.*)"\]$/)) { + my $arg = $1; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + my $t = $self->translate($arg, + $self->{ref}, + "icon", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline("[icon=\"$t\"]\n"); + @comments=(); + $wrapped_mode = 1; + undef $self->{bullet}; + undef $self->{indent}; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^\[icons=None, +caption="(.*)"\]$/)) { + my $arg = $1; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + my $t = $self->translate($arg, + $self->{ref}, + "caption", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline("[icons=None, caption=\"$t\"]\n"); + @comments=(); + $wrapped_mode = 1; + undef $self->{bullet}; + undef $self->{indent}; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^(\s*)([*_+`'#[:alnum:]].*)((?:::|;;|\?\?|:-)(?: *\\)?)$/)) { + my $indent = $1; + my $label = $2; + my $labelend = $3; + # Found labeled list + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $self->{bullet} = ""; + $self->{indent} = $indent; + my $t = $self->translate($label, + $self->{ref}, + "Labeled list", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline("$indent$t$labelend\n"); + @comments=(); + } elsif (not defined $self->{verbatim} and + ($line =~ m/^(\s*)(\S.*)((?:::|;;)\s+)(.*)$/)) { + my $indent = $1; + my $label = $2; + my $labelend = $3; + my $labeltext = $4; + # Found Horizontal Labeled Lists + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=$labeltext."\n"; + $wrapped_mode = 1; + $self->{bullet} = ""; + $self->{indent} = $indent; + my $t = $self->translate($label, + $self->{ref}, + "Labeled list", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline("$indent$t$labelend"); + @comments=(); + } elsif (not defined $self->{verbatim} and + ($line =~ m/^\:(\S.*?)(:\s*)(.*)$/)) { + my $attrname = $1; + my $attrsep = $2; + my $attrvalue = $3; + # Found a Attribute entry + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + undef $self->{bullet}; + undef $self->{indent}; + my $t = $self->translate($attrvalue, + $self->{ref}, + "Attribute :$attrname:", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline(":$attrname$attrsep$t\n"); + @comments=(); + } elsif (not defined $self->{verbatim} and + ($line !~ m/^\.\./) and ($line =~ m/^\.(\S.*)$/)) { + my $title = $1; + # Found block title + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + undef $self->{bullet}; + undef $self->{indent}; + my $t = $self->translate($title, + $self->{ref}, + "Block title", + "comment" => join("\n", @comments), + "wrap" => 0); + $self->pushline(".$t\n"); + @comments=(); + } elsif (not defined $self->{verbatim} and + ($line =~ m/^(\s*)((?:[-*o+]|(?:[0-9]+[.\)])|(?:[a-z][.\)])|\([0-9]+\)|\.|\.\.)\s+)(.*)$/)) { + my $indent = $1||""; + my $bullet = $2; + my $text = $3; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph = $text."\n"; + $self->{indent} = $indent; + $self->{bullet} = $bullet; + } elsif (not defined $self->{verbatim} and + ($line =~ m/^((?:<?[0-9]+)?> +)(.*)$/)) { + my $bullet = $1; + my $text = $2; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph = $text."\n"; + $self->{indent} = ""; + $self->{bullet} = $bullet; + } elsif (not defined $self->{verbatim} and + (defined $self->{bullet} and $line =~ m/^(\s+)(.*)$/)) { + my $indent = $1; + my $text = $2; + if (not defined $self->{indent}) { + $paragraph .= $text."\n"; + $self->{indent} = $indent; + } elsif (length($paragraph) and (length($self->{bullet}) + length($self->{indent}) == length($indent))) { + $paragraph .= $text."\n"; + } else { + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph = $text."\n"; + $self->{indent} = $indent; + $self->{bullet} = ""; + } + } else { + return parse_fallback($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); + } + return ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); +} + +sub parse_markdown { + my ($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = @_; + if (($line =~ m/^(={4,}|-{4,})$/) and + (defined($paragraph) ) and + ($paragraph =~ m/^[^\n]*\n$/s) and + (length($paragraph) == (length($line)+1))) { + # XXX: There can be any number of underlining according + # to the documentation. This detection, which avoid + # translating the formatting, is only supported if + # the underlining has the same size as the header text. + # Found title + $wrapped_mode = 0; + my $level = $line; + $level =~ s/^(.).*$/$1/; + my $t = $self->translate($paragraph, + $self->{ref}, + "Title $level", + "wrap" => 0); + $self->pushline($t); + $paragraph=""; + $wrapped_mode = 1; + $self->pushline(($level x (length($t)-1))."\n"); + } elsif ($line =~ m/^(#{1,6})( +)(.*?)( +\1)?$/) { + my $titlelevel1 = $1; + my $titlespaces = $2; + my $title = $3; + my $titlelevel2 = $4||""; + # Found one line title + do_paragraph($self,$paragraph,$wrapped_mode); + $wrapped_mode = 0; + $paragraph=""; + my $t = $self->translate($title, + $self->{ref}, + "Title $titlelevel1", + "wrap" => 0); + $self->pushline($titlelevel1.$titlespaces.$t.$titlelevel2."\n"); + $wrapped_mode = 1; + } elsif (($paragraph eq "") and + ($line =~ /^((\*\s*){3,}|(-\s*){3,}|(_\s*){3,})$/)) { + # Horizontal rule + $wrapped_mode = 1; + $self->pushline($line."\n"); + } elsif ( $line =~ /^\s*\[\[\!\S+\s*$/ # macro begin + or $line =~ /^\s*"""\s*\]\]\s*$/) { # """ textblock inside macro end + # Avoid translating Markdown lines containing only markup + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $self->pushline("$line\n"); + } elsif ( $line =~ /^#/ # headline + or $line =~ /^\s*\[\[\!\S[^\]]*\]\]\s*$/) { # sole macro + # Preserve some Markdown markup as a single line + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph="$line\n"; + $wrapped_mode = 0; + $end_of_paragraph = 1; + } elsif ($line =~ /^"""/) { # """ textblock inside macro end + # Markdown markup needing separation _before_ this line + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph="$line\n"; + $wrapped_mode = 1; + } else { + return parse_fallback($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); + } + return ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); +} + +sub parse { + my $self = shift; + my ($line,$ref); + my $paragraph=""; + my $wrapped_mode = 1; + my $expect_header = 1; + my $end_of_paragraph = 0; + ($line,$ref)=$self->shiftline(); + my $file = $ref; + $file =~ s/:[0-9]+$// if defined($line); + while (defined($line)) { + $ref =~ m/^(.*):[0-9]+$/; + if ($1 ne $file) { + $file = $1; + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $expect_header = 1; + } + + chomp($line); + $self->{ref}="$ref"; + ($paragraph,$wrapped_mode,$expect_header,$end_of_paragraph) = &$parse_func($self,$line,$ref,$paragraph,$wrapped_mode,$expect_header,$end_of_paragraph); + # paragraphs starting by a bullet, or numbered + # or paragraphs with a line containing many consecutive spaces + # (more than 3) + # are considered as verbatim paragraphs + $wrapped_mode = 0 if ( $paragraph =~ m/^(\*|[0-9]+[.)] )/s + or $paragraph =~ m/[ \t][ \t][ \t]/s); + $wrapped_mode = 0 if ( $tabs eq "verbatim" + and $paragraph =~ m/\t/s); + if ($markdown) { + # Some Markdown markup can (or might) not survive wrapping + $wrapped_mode = 0 if ( + $paragraph =~ /^>/ms # blockquote + or $paragraph =~ /^( {8}|\t)/ms # monospaced + or $paragraph =~ /^\$(\S+[{}]\S*\s*)+/ms # Xapian macro + or $paragraph =~ /<(?![a-z]+[:@])/ms # maybe html (tags but not wiki <URI>) + or $paragraph =~ /^[^<]+>/ms # maybe html (tag with vertical space) + or $paragraph =~ /\S $/ms # explicit newline + or $paragraph =~ /\[\[\!\S[^\]]+$/ms # macro begin + ); + } + if ($end_of_paragraph) { + do_paragraph($self,$paragraph,$wrapped_mode); + $paragraph=""; + $wrapped_mode = 1; + $end_of_paragraph = 0; + } + ($line,$ref)=$self->shiftline(); + } + if (length $paragraph) { + do_paragraph($self,$paragraph,$wrapped_mode); + } +} + +sub do_paragraph { + my ($self, $paragraph, $wrap) = (shift, shift, shift); + my $type = shift || $self->{type} || "Plain text"; + return if ($paragraph eq ""); + +# DEBUG +# my $b; +# if (defined $self->{bullet}) { +# $b = $self->{bullet}; +# } else { +# $b = "UNDEF"; +# } +# $type .= " verbatim: '".($self->{verbatim}||"NONE")."' bullet: '$b' indent: '".($self->{indent}||"NONE")."' type: '".($self->{type}||"NONE")."'"; + + if ($bullets and not $wrap and not defined $self->{verbatim}) { + # Detect bullets + # | * blah blah + # |<spaces> blah + # | ^-- aligned + # <empty line> + # + # Other bullets supported: + # - blah o blah + blah + # 1. blah 1) blah (1) blah +TEST_BULLET: + if ($paragraph =~ m/^(\s*)((?:[-*o+]|([0-9]+[.\)])|\([0-9]+\))\s+)([^\n]*\n)(.*)$/s) { + my $para = $5; + my $bullet = $2; + my $indent1 = $1; + my $indent2 = "$1".(' ' x length $bullet); + my $text = $4; + while ($para !~ m/$indent2(?:[-*o+]|([0-9]+[.\)])|\([0-9]+\))\s+/ + and $para =~ s/^$indent2(\S[^\n]*\n)//s) { + $text .= $1; + } + # TODO: detect if a line starts with the same bullet + if ($text !~ m/\S[ \t][ \t][ \t]+\S/s) { + my $bullet_regex = quotemeta($indent1.$bullet); + $bullet_regex =~ s/[0-9]+/\\d\+/; + if ($para eq '' or $para =~ m/^$bullet_regex\S/s) { + my $trans = $self->translate($text, + $self->{ref}, + "Bullet: '$indent1$bullet'", + "wrap" => 1, + "wrapcol" => - (length $indent2)); + $trans =~ s/^/$indent1$bullet/s; + $trans =~ s/\n(.)/\n$indent2$1/sg; + $self->pushline( $trans."\n" ); + if ($para eq '') { + return; + } else { + # Another bullet + $paragraph = $para; + goto TEST_BULLET; + } + } + } + } + } + + my $end = ""; + if ($wrap) { + $paragraph =~ s/^(.*?)(\n*)$/$1/s; + $end = $2 || ""; + } + my $t = $self->translate($paragraph, + $self->{ref}, + $type, + "comment" => join("\n", @comments), + "wrap" => $wrap); + @comments = (); + if (defined $self->{bullet}) { + my $bullet = $self->{bullet}; + my $indent1 = $self->{indent}; + my $indent2 = $indent1.(' ' x length($bullet)); + $t =~ s/^/$indent1$bullet/s; + $t =~ s/\n(.)/\n$indent2$1/sg; + } + $self->pushline( $t.$end ); +} + +1; + +=head1 STATUS OF THIS MODULE + +Tested successfully on simple text files and NEWS.Debian files. + +=head1 AUTHORS + + Nicolas François <nicolas.francois@centraliens.net> + +=head1 COPYRIGHT AND LICENSE + + Copyright 2005-2008 by Nicolas FRANÇOIS <nicolas.francois@centraliens.net>. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/TransTractor.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/TransTractor.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/TransTractor.pm (revision 2840) @@ -0,0 +1,1123 @@ +#!/usr/bin/perl -w + +require Exporter; + +package Locale::Po4a::TransTractor; +use DynaLoader; + +use 5.006; +use strict; +use warnings; + +use subs qw(makespace); +use vars qw($VERSION @ISA @EXPORT); +$VERSION="0.44"; +@ISA = qw(DynaLoader); +@EXPORT = qw(new process translate + read write readpo writepo + getpoout setpoout get_out_charset); + +# Try to use a C extension if present. +eval("bootstrap Locale::Po4a::TransTractor $VERSION"); + +use Carp qw(croak); +use Locale::Po4a::Po; +use Locale::Po4a::Common; + +use File::Path; # mkdir before write + +use Encode; +use Encode::Guess; + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::TransTractor - generic trans(lator ex)tractor. + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +This class is the ancestor of every po4a parser used to parse a document, to +search translatable strings, to extract them to a PO file and to replace them by +their translation in the output document. + +More formally, it takes the following arguments as input: + +=over 2 + +=item - + +a document to translate; + +=item - + +a PO file containing the translations to use. + +=back + +As output, it produces: + +=over 2 + +=item - + +another PO file, resulting of the extraction of translatable strings from +the input document; + +=item - + +a translated document, with the same structure than the one in input, but +with all translatable strings replaced with the translations found in the +PO file provided in input. + +=back + +Here is a graphical representation of this: + + Input document --\ /---> Output document + \ / (translated) + +-> parse() function -----+ + / \ + Input PO --------/ \---> Output PO + (extracted) + +=head1 FUNCTIONS YOUR PARSER SHOULD OVERRIDE + +=over 4 + +=item parse() + +This is where all the work takes place: the parsing of input documents, the +generation of output, and the extraction of the translatable strings. This +is pretty simple using the provided functions presented in the section +B<INTERNAL FUNCTIONS> below. See also the B<SYNOPSIS>, which presents an +example. + +This function is called by the process() function below, but if you choose +to use the new() function, and to add content manually to your document, +you will have to call this function yourself. + +=item docheader() + +This function returns the header we should add to the produced document, +quoted properly to be a comment in the target language. See the section +B<Educating developers about translations>, from L<po4a(7)|po4a.7>, for what +it is good for. + +=back + +=cut + +sub docheader {} + +sub parse {} + +=head1 SYNOPSIS + +The following example parses a list of paragraphs beginning with "<p>". For the sake +of simplicity, we assume that the document is well formatted, i.e. that '<p>' +tags are the only tags present, and that this tag is at the very beginning +of each paragraph. + + sub parse { + my $self = shift; + + PARAGRAPH: while (1) { + my ($paragraph,$pararef)=("",""); + my $first=1; + my ($line,$lref)=$self->shiftline(); + while (defined($line)) { + if ($line =~ m/<p>/ && !$first--; ) { + # Not the first time we see <p>. + # Reput the current line in input, + # and put the built paragraph to output + $self->unshiftline($line,$lref); + + # Now that the document is formed, translate it: + # - Remove the leading tag + $paragraph =~ s/^<p>//s; + + # - push to output the leading tag (untranslated) and the + # rest of the paragraph (translated) + $self->pushline( "<p>" + . $document->translate($paragraph,$pararef) + ); + + next PARAGRAPH; + } else { + # Append to the paragraph + $paragraph .= $line; + $pararef = $lref unless(length($pararef)); + } + + # Reinit the loop + ($line,$lref)=$self->shiftline(); + } + # Did not get a defined line? End of input file. + return; + } + } + +Once you've implemented the parse function, you can use your document +class, using the public interface presented in the next section. + +=head1 PUBLIC INTERFACE for scripts using your parser + +=head2 Constructor + +=over 4 + +=item process(%) + +This function can do all you need to do with a po4a document in one +invocation. Its arguments must be packed as a hash. ACTIONS: + +=over 3 + +=item a. + +Reads all the PO files specified in po_in_name + +=item b. + +Reads all original documents specified in file_in_name + +=item c. + +Parses the document + +=item d. + +Reads and applies all the addenda specified + +=item e. + +Writes the translated document to file_out_name (if given) + +=item f. + +Writes the extracted PO file to po_out_name (if given) + +=back + +ARGUMENTS, beside the ones accepted by new() (with expected type): + +=over 4 + +=item file_in_name (@) + +List of filenames where we should read the input document. + +=item file_in_charset ($) + +Charset used in the input document (if it isn't specified, it will try +to detect it from the input document). + +=item file_out_name ($) + +Filename where we should write the output document. + +=item file_out_charset ($) + +Charset used in the output document (if it isn't specified, it will use +the PO file charset). + +=item po_in_name (@) + +List of filenames where we should read the input PO files from, containing +the translation which will be used to translate the document. + +=item po_out_name ($) + +Filename where we should write the output PO file, containing the strings +extracted from the input document. + +=item addendum (@) + +List of filenames where we should read the addenda from. + +=item addendum_charset ($) + +Charset for the addenda. + +=back + +=item new(%) + +Create a new po4a document. Accepted options (but be in a hash): + +=over 4 + +=item verbose ($) + +Sets the verbosity. + +=item debug ($) + +Sets the debugging. + +=back + +=cut + +sub process { + ## Determine if we were called via an object-ref or a classname + my $self = shift; + + ## Any remaining arguments are treated as initial values for the + ## hash that is used to represent this object. + my %params = @_; + + # Build the args for new() + my %newparams = (); + foreach (keys %params) { + next if ($_ eq 'po_in_name' || + $_ eq 'po_out_name' || + $_ eq 'file_in_name' || + $_ eq 'file_in_charset' || + $_ eq 'file_out_name' || + $_ eq 'file_out_charset' || + $_ eq 'addendum' || + $_ eq 'addendum_charset'); + $newparams{$_}=$params{$_}; + } + + $self->detected_charset($params{'file_in_charset'}); + $self->{TT}{'file_out_charset'}=$params{'file_out_charset'}; + if (defined($self->{TT}{'file_out_charset'}) and + length($self->{TT}{'file_out_charset'})) { + $self->{TT}{'file_out_encoder'} = find_encoding($self->{TT}{'file_out_charset'}); + } + $self->{TT}{'addendum_charset'}=$params{'addendum_charset'}; + + chdir $params{'srcdir'} + if (defined $params{'srcdir'}); + foreach my $file (@{$params{'po_in_name'}}) { + print STDERR "readpo($file)... " if $self->debug(); + $self->readpo($file); + print STDERR "done.\n" if $self->debug() + } + foreach my $file (@{$params{'file_in_name'}}) { + print STDERR "read($file)..." if $self->debug(); + $self->read($file); + print STDERR "done.\n" if $self->debug(); + } + print STDERR "parse..." if $self->debug(); + $self->parse(); + print STDERR "done.\n" if $self->debug(); + foreach my $file (@{$params{'addendum'}}) { + print STDERR "addendum($file)..." if $self->debug(); + $self->addendum($file) || die "An addendum failed\n"; + print STDERR "done.\n" if $self->debug(); + } + chdir $params{'destdir'} + if (defined $params{'destdir'}); + if (defined $params{'file_out_name'}) { + print STDERR "write(".$params{'file_out_name'}.")... " + if $self->debug(); + $self->write($params{'file_out_name'}); + print STDERR "done.\n" if $self->debug(); + } + chdir $params{'srcdir'} + if (defined $params{'srcdir'}); + if (defined $params{'po_out_name'}) { + print STDERR "writepo(".$params{'po_out_name'}.")... " + if $self->debug(); + $self->writepo($params{'po_out_name'}); + print STDERR "done.\n" if $self->debug(); + } + chdir $params{'calldir'} + if (defined $params{'calldir'}); + return $self; +} + +sub new { + ## Determine if we were called via an object-ref or a classname + my $this = shift; + my $class = ref($this) || $this; + my $self = { }; + my %options=@_; + ## Bless ourselves into the desired class and perform any initialization + bless $self, $class; + + ## initialize the plugin + # prevent the plugin from croaking on the options intended for Po.pm + $self->{options}{'porefs'} = ''; + $self->{options}{'copyright-holder'} = ''; + $self->{options}{'msgid-bugs-address'} = ''; + $self->{options}{'package-name'} = ''; + $self->{options}{'package-version'} = ''; + # let the plugin parse the options and such + $self->initialize(%options); + + ## Create our private data + my %po_options; + $po_options{'porefs'} = $self->{options}{'porefs'}; + $po_options{'copyright-holder'} = $options{'copyright-holder'}; + $po_options{'msgid-bugs-address'} = $options{'msgid-bugs-address'}; + $po_options{'package-name'} = $options{'package-name'}; + $po_options{'package-version'} = $options{'package-version'}; + + # private data + $self->{TT}=(); + $self->{TT}{po_in}=Locale::Po4a::Po->new(\%po_options); + $self->{TT}{po_out}=Locale::Po4a::Po->new(\%po_options); + # Warning, this is an array of array: + # The document is splited on lines, and for each + # [0] is the line content, [1] is the reference [2] the type + $self->{TT}{doc_in}=(); + $self->{TT}{doc_out}=(); + if (defined $options{'verbose'}) { + $self->{TT}{verbose} = $options{'verbose'}; + } + if (defined $options{'debug'}) { + $self->{TT}{debug} = $options{'debug'}; + } + # Input document is in ascii until we prove the opposite (in read()) + $self->{TT}{ascii_input}=1; + # We try not to use utf unless it's forced from the outside (in case the + # document isn't in ascii) + $self->{TT}{utf_mode}=0; + + return $self; +} + +=back + +=head2 Manipulating document files + +=over 4 + +=item read($) + +Add another input document at the end of the existing one. The argument is +the filename to read. + +Please note that it does not parse anything. You should use the parse() +function when you're done with packing input files into the document. + +=cut + +#' +sub read() { + my $self=shift; + my $filename=shift + or croak wrap_msg(dgettext("po4a", "Can't read from file without having a filename")); + my $linenum=0; + + open INPUT,"<$filename" + or croak wrap_msg(dgettext("po4a", "Can't read from %s: %s"), $filename, $!); + while (defined (my $textline = <INPUT>)) { + $linenum++; + my $ref="$filename:$linenum"; + $textline =~ s/\r$//; + my @entry=($textline,$ref); + push @{$self->{TT}{doc_in}}, @entry; + + if (!defined($self->{TT}{'file_in_charset'})) { + # Detect if this file has non-ascii characters + if($self->{TT}{ascii_input}) { + my $decoder = guess_encoding($textline); + if (!ref($decoder) or $decoder !~ /Encode::XS=/) { + # We have detected a non-ascii line + $self->{TT}{ascii_input} = 0; + # Save the reference for future error message + $self->{TT}{non_ascii_ref} ||= $ref; + } + } + } + } + close INPUT + or croak wrap_msg(dgettext("po4a", "Can't close %s after reading: %s"), $filename, $!); + +} + +=item write($) + +Write the translated document to the given filename. + +=cut + +sub write { + my $self=shift; + my $filename=shift + or croak wrap_msg(dgettext("po4a", "Can't write to a file without filename")); + + my $fh; + if ($filename eq '-') { + $fh=\*STDOUT; + } else { + # make sure the directory in which we should write the localized file exists + my $dir = $filename; + if ($dir =~ m|/|) { + $dir =~ s|/[^/]*$||; + + File::Path::mkpath($dir, 0, 0755) # Croaks on error + if (length ($dir) && ! -e $dir); + } + open $fh,">$filename" + or croak wrap_msg(dgettext("po4a", "Can't write to %s: %s"), $filename, $!); + } + + map { print $fh $_ } $self->docheader(); + map { print $fh $_ } @{$self->{TT}{doc_out}}; + + if ($filename ne '-') { + close $fh or croak wrap_msg(dgettext("po4a", "Can't close %s after writing: %s"), $filename, $!); + } + +} + +=back + +=head2 Manipulating PO files + +=over 4 + +=item readpo($) + +Add the content of a file (which name is passed as argument) to the +existing input PO. The old content is not discarded. + +=item writepo($) + +Write the extracted PO file to the given filename. + +=item stats() + +Returns some statistics about the translation done so far. Please note that +it's not the same statistics than the one printed by msgfmt +--statistic. Here, it's stats about recent usage of the PO file, while +msgfmt reports the status of the file. It is a wrapper to the +Locale::Po4a::Po::stats_get function applied to the input PO file. Example +of use: + + [normal use of the po4a document...] + + ($percent,$hit,$queries) = $document->stats(); + print "We found translations for $percent\% ($hit from $queries) of strings.\n"; + +=back + +=cut + +sub getpoout { + return $_[0]->{TT}{po_out}; +} +sub setpoout { + $_[0]->{TT}{po_out} = $_[1]; +} +sub readpo { + $_[0]->{TT}{po_in}->read($_[1]); +} +sub writepo { + $_[0]->{TT}{po_out}->write( $_[1] ); +} +sub stats { + return $_[0]->{TT}{po_in}->stats_get(); +} + +=head2 Manipulating addenda + +=over 4 + +=item addendum($) + +Please refer to L<po4a(7)|po4a.7> for more information on what addenda are, +and how translators should write them. To apply an addendum to the translated +document, simply pass its filename to this function and you are done ;) + +This function returns a non-null integer on error. + +=cut + +# Internal function to read the header. +sub addendum_parse { + my ($filename,$header)=shift; + + my ($errcode,$mode,$position,$boundary,$bmode,$content)= + (1,"","","","",""); + + unless (open (INS, "<$filename")) { + warn wrap_msg(dgettext("po4a", "Can't read from %s: %s"), $filename, $!); + goto END_PARSE_ADDFILE; + } + + unless (defined ($header=<INS>) && $header) { + warn wrap_msg(dgettext("po4a", "Can't read po4a header from %s."), $filename); + goto END_PARSE_ADDFILE; + } + + unless ($header =~ s/PO4A-HEADER://i) { + warn wrap_msg(dgettext("po4a", "First line of %s does not look like a po4a header."), $filename); + goto END_PARSE_ADDFILE; + } + foreach my $part (split(/;/,$header)) { + unless ($part =~ m/^\s*([^=]*)=(.*)$/) { + warn wrap_msg(dgettext("po4a", "Syntax error in po4a header of %s, near \"%s\""), $filename, $part); + goto END_PARSE_ADDFILE; + } + my ($key,$value)=($1,$2); + $key=lc($key); + if ($key eq 'mode') { + $mode=lc($value); + } elsif ($key eq 'position') { + $position=$value; + } elsif ($key eq 'endboundary') { + $boundary=$value; + $bmode='after'; + } elsif ($key eq 'beginboundary') { + $boundary=$value; + $bmode='before'; + } else { + warn wrap_msg(dgettext("po4a", "Invalid argument in the po4a header of %s: %s"), $filename, $key); + goto END_PARSE_ADDFILE; + } + } + + unless (length($mode)) { + warn wrap_msg(dgettext("po4a", "The po4a header of %s does not define the mode."), $filename); + goto END_PARSE_ADDFILE; + } + unless ($mode eq "before" || $mode eq "after") { + warn wrap_msg(dgettext("po4a", "Mode invalid in the po4a header of %s: should be 'before' or 'after' not %s."), $filename, $mode); + goto END_PARSE_ADDFILE; + } + + unless (length($position)) { + warn wrap_msg(dgettext("po4a", "The po4a header of %s does not define the position."), $filename); + goto END_PARSE_ADDFILE; + } + unless ($mode eq "before" || length($boundary)) { + warn wrap_msg(dgettext("po4a", "No ending boundary given in the po4a header, but mode=after.")); + goto END_PARSE_ADDFILE; + } + + while (defined(my $line = <INS>)) { + $content .= $line; + } + close INS; + + $errcode=0; + END_PARSE_ADDFILE: + return ($errcode,$mode,$position,$boundary,$bmode,$content); +} + +sub mychomp { + my ($str) = shift; + chomp($str); + return $str; +} + +sub addendum { + my ($self,$filename) = @_; + + print STDERR "Apply addendum $filename..." if $self->debug(); + unless ($filename) { + warn wrap_msg(dgettext("po4a", + "Can't apply addendum when not given the filename")); + return 0; + } + die wrap_msg(dgettext("po4a", "Addendum %s does not exist."), $filename) + unless -e $filename; + + my ($errcode,$mode,$position,$boundary,$bmode,$content)= + addendum_parse($filename); + return 0 if ($errcode); + + print STDERR "mode=$mode;pos=$position;bound=$boundary;bmode=$bmode;ctn=$content\n" + if $self->debug(); + + # We only recode the addendum if an origin charset is specified, else we + # suppose it's already in the output document's charset + if (defined($self->{TT}{'addendum_charset'}) && + length($self->{TT}{'addendum_charset'})) { + Encode::from_to($content,$self->{TT}{'addendum_charset'}, + $self->get_out_charset); + } + + my $found = scalar grep { /$position/ } @{$self->{TT}{doc_out}}; + if ($found == 0) { + warn wrap_msg(dgettext("po4a", + "No candidate position for the addendum %s."), $filename); + return 0; + } + if ($found > 1) { + warn wrap_msg(dgettext("po4a", + "More than one candidate position found for the addendum %s."), $filename); + return 0; + } + + if ($mode eq "before") { + if ($self->verbose() > 1 || $self->debug() ) { + map { print STDERR wrap_msg(dgettext("po4a", "Addendum '%s' applied before this line: %s"), $filename, $_) if (/$position/); + } @{$self->{TT}{doc_out}}; + } + @{$self->{TT}{doc_out}} = map { /$position/ ? ($content,$_) : $_ + } @{$self->{TT}{doc_out}}; + } else { + my @newres=(); + + do { + # make sure it doesn't whine on empty document + my $line = scalar @{$self->{TT}{doc_out}} ? shift @{$self->{TT}{doc_out}} : ""; + push @newres,$line; + my $outline=mychomp($line); + $outline =~ s/^[ \t]*//; + + if ($line =~ m/$position/) { + while ($line=shift @{$self->{TT}{doc_out}}) { + last if ($line=~/$boundary/); + push @newres,$line; + } + if (defined $line) { + if ($bmode eq 'before') { + print wrap_msg(dgettext("po4a", + "Addendum '%s' applied before this line: %s"), + $filename, $outline) + if ($self->verbose() > 1 || $self->debug()); + push @newres,$content; + push @newres,$line; + } else { + print wrap_msg(dgettext("po4a", + "Addendum '%s' applied after the line: %s."), + $filename, $outline) + if ($self->verbose() > 1 || $self->debug()); + push @newres,$line; + push @newres,$content; + } + } else { + print wrap_msg(dgettext("po4a", "Addendum '%s' applied at the end of the file."), $filename) + if ($self->verbose() > 1 || $self->debug()); + push @newres,$content; + } + } + } while (scalar @{$self->{TT}{doc_out}}); + @{$self->{TT}{doc_out}} = @newres; + } + print STDERR "done.\n" if $self->debug(); + return 1; +} + +=back + +=head1 INTERNAL FUNCTIONS used to write derivated parsers + +=head2 Getting input, providing output + +Four functions are provided to get input and return output. They are very +similar to shift/unshift and push/pop. The first pair is about input, while +the second is about output. Mnemonic: in input, you are interested in the +first line, what shift gives, and in output you want to add your result at +the end, like push does. + +=over 4 + +=item shiftline() + +This function returns the next line of the doc_in to be parsed and its +reference (packed as an array). + +=item unshiftline($$) + +Unshifts a line of the input document and its reference. + +=item pushline($) + +Push a new line to the doc_out. + +=item popline() + +Pop the last pushed line from the doc_out. + +=back + +=cut + +sub shiftline { + my ($line,$ref)=(shift @{$_[0]->{TT}{doc_in}}, + shift @{$_[0]->{TT}{doc_in}}); + return ($line,$ref); +} +sub unshiftline { + my $self = shift; + unshift @{$self->{TT}{doc_in}},@_; +} + +sub pushline { push @{$_[0]->{TT}{doc_out}}, $_[1] if defined $_[1]; } +sub popline { return pop @{$_[0]->{TT}{doc_out}}; } + +=head2 Marking strings as translatable + +One function is provided to handle the text which should be translated. + +=over 4 + +=item translate($$$) + +Mandatory arguments: + +=over 2 + +=item - + +A string to translate + +=item - + +The reference of this string (i.e. position in inputfile) + +=item - + +The type of this string (i.e. the textual description of its structural role; +used in Locale::Po4a::Po::gettextization(); see also L<po4a(7)|po4a.7>, +section B<Gettextization: how does it work?>) + +=back + +This function can also take some extra arguments. They must be organized as +a hash. For example: + + $self->translate("string","ref","type", + 'wrap' => 1); + +=over + +=item B<wrap> + +boolean indicating whether we can consider that whitespaces in string are +not important. If yes, the function canonizes the string before looking for +a translation or extracting it, and wraps the translation. + +=item B<wrapcol> + +the column at which we should wrap (default: 76). + +=item B<comment> + +an extra comment to add to the entry. + +=back + +Actions: + +=over 2 + +=item - + +Pushes the string, reference and type to po_out. + +=item - + +Returns the translation of the string (as found in po_in) so that the +parser can build the doc_out. + +=item - + +Handles the charsets to recode the strings before sending them to +po_out and before returning the translations. + +=back + +=back + +=cut + +sub translate { + my $self=shift; + my ($string,$ref,$type)=(shift,shift,shift); + my (%options)=@_; + + # my $validoption="wrap wrapcol"; + # my %validoption; + + return "" unless defined($string) && length($string); + + # map { $validoption{$_}=1 } (split(/ /,$validoption)); + # foreach (keys %options) { + # Carp::confess "internal error: translate() called with unknown arg $_. Valid options: $validoption" + # unless $validoption{$_}; + # } + + my $in_charset; + if ($self->{TT}{ascii_input}) { + $in_charset = "ascii"; + } else { + if (defined($self->{TT}{'file_in_charset'}) and + length($self->{TT}{'file_in_charset'}) and + $self->{TT}{'file_in_charset'} !~ m/ascii/i) { + $in_charset=$self->{TT}{'file_in_charset'}; + } else { + # FYI, the document charset have to be determined *before* we see the first + # string to recode. + die wrap_mod("po4a", dgettext("po4a", "Couldn't determine the input document's charset. Please specify it on the command line. (non-ASCII char at %s)"), $self->{TT}{non_ascii_ref}) + } + } + + if ($self->{TT}{po_in}->get_charset ne "CHARSET") { + $string = encode_from_to($string, + $self->{TT}{'file_in_encoder'}, + $self->{TT}{po_in}{encoder}); + } + + if (defined $options{'wrapcol'} && $options{'wrapcol'} < 0) { +# FIXME: should be the parameter given with --width + $options{'wrapcol'} = 76 + $options{'wrapcol'}; + } + my $transstring = $self->{TT}{po_in}->gettext($string, + 'wrap' => $options{'wrap'}||0, + 'wrapcol' => $options{'wrapcol'}); + + if ($self->{TT}{po_in}->get_charset ne "CHARSET") { + my $out_encoder = $self->{TT}{'file_out_encoder'}; + unless (defined $out_encoder) { + $out_encoder = find_encoding($self->get_out_charset) + } + $transstring = encode_from_to($transstring, + $self->{TT}{po_in}{encoder}, + $out_encoder); + } + + # If the input document isn't completely in ascii, we should see what to + # do with the current string + unless ($self->{TT}{ascii_input}) { + my $out_charset = $self->{TT}{po_out}->get_charset; + # We set the output po charset + if ($out_charset eq "CHARSET") { + if ($self->{TT}{utf_mode}) { + $out_charset="UTF-8"; + } else { + $out_charset=$in_charset; + } + $self->{TT}{po_out}->set_charset($out_charset); + } + if ( $in_charset !~ /^$out_charset$/i ) { + Encode::from_to($string,$in_charset,$out_charset); + if (defined($options{'comment'}) and length($options{'comment'})) { + Encode::from_to($options{'comment'},$in_charset,$out_charset); + } + } + } + + # the comments provided by the modules are automatic comments from the PO point of view + $self->{TT}{po_out}->push('msgid' => $string, + 'reference' => $ref, + 'type' => $type, + 'automatic' => $options{'comment'}, + 'wrap' => $options{'wrap'}||0, + 'wrapcol' => $options{'wrapcol'}); + +# if ($self->{TT}{po_in}->get_charset ne "CHARSET") { +# Encode::from_to($transstring,$self->{TT}{po_in}->get_charset, +# $self->get_out_charset); +# } + + if ($options{'wrap'}||0) { + $transstring =~ s/( *)$//s; + my $trailing_spaces = $1||""; + $transstring =~ s/(?<!\\) +$//gm; + $transstring .= $trailing_spaces; + } + + return $transstring; +} + +=head2 Misc functions + +=over 4 + +=item verbose() + +Returns if the verbose option was passed during the creation of the +TransTractor. + +=cut + +sub verbose { + if (defined $_[1]) { + $_[0]->{TT}{verbose} = $_[1]; + } else { + return $_[0]->{TT}{verbose} || 0; # undef and 0 have the same meaning, but one generates warnings + } +} + +=item debug() + +Returns if the debug option was passed during the creation of the +TransTractor. + +=cut + +sub debug { + return $_[0]->{TT}{debug}; +} + +=item detected_charset($) + +This tells TransTractor that a new charset (the first argument) has been +detected from the input document. It can usually be read from the document +header. Only the first charset will remain, coming either from the +process() arguments or detected from the document. + +=cut + +sub detected_charset { + my ($self,$charset)=(shift,shift); + unless (defined($self->{TT}{'file_in_charset'}) and + length($self->{TT}{'file_in_charset'}) ) { + $self->{TT}{'file_in_charset'}=$charset; + if (defined $charset) { + $self->{TT}{'file_in_encoder'}=find_encoding($charset); + } else { + $self->{TT}{ascii_input}=1; + $self->{TT}{utf_mode}=0; + } + } + + if (defined $self->{TT}{'file_in_charset'} and + length $self->{TT}{'file_in_charset'} and + $self->{TT}{'file_in_charset'} !~ m/ascii/i) { + $self->{TT}{ascii_input}=0; + } +} + +=item get_out_charset() + +This function will return the charset that should be used in the output +document (usually useful to substitute the input document's detected charset +where it has been found). + +It will use the output charset specified in the command line. If it wasn't +specified, it will use the input PO's charset, and if the input PO has the +default "CHARSET", it will return the input document's charset, so that no +encoding is performed. + +=cut + +sub get_out_charset { + my $self=shift; + my $charset; + + # Use the value specified at the command line + if (defined($self->{TT}{'file_out_charset'}) and + length($self->{TT}{'file_out_charset'})) { + $charset=$self->{TT}{'file_out_charset'}; + } else { + if ($self->{TT}{utf_mode} && $self->{TT}{ascii_input}) { + $charset="UTF-8"; + } else { + $charset=$self->{TT}{po_in}->get_charset; + $charset=$self->{TT}{'file_in_charset'} + if $charset eq "CHARSET" and + defined($self->{TT}{'file_in_charset'}) and + length($self->{TT}{'file_in_charset'}); + $charset="ascii" + if $charset eq "CHARSET"; + } + } + return $charset; +} + +=item recode_skipped_text($) + +This function returns the recoded text passed as argument, from the input +document's charset to the output document's one. This isn't needed when +translating a string (translate() recodes everything itself), but it is when +you skip a string from the input document and you want the output document to +be consistent with the global encoding. + +=cut + +sub recode_skipped_text { + my ($self,$text)=(shift,shift); + unless ($self->{TT}{'ascii_input'}) { + if(defined($self->{TT}{'file_in_charset'}) and + length($self->{TT}{'file_in_charset'}) ) { + $text = encode_from_to($text, + $self->{TT}{'file_in_encoder'}, + find_encoding($self->get_out_charset)); + } else { + die wrap_mod("po4a", dgettext("po4a", "Couldn't determine the input document's charset. Please specify it on the command line. (non-ASCII char at %s)"), $self->{TT}{non_ascii_ref}) + } + } + return $text; +} + + +# encode_from_to($,$,$) +# +# Encode the given text from one encoding to another one. +# It differs from Encode::from_to because it does not take the name of the +# encoding in argument, but the encoders (as returned by the +# Encode::find_encoding(<name>) method). Thus it permits to save a bunch +# of call to find_encoding. +# +# If the "from" encoding is undefined, it is considered as UTF-8 (or +# ascii). +# If the "to" encoding is undefined, it is considered as UTF-8. +# +sub encode_from_to { + my ($text,$from,$to) = (shift,shift,shift); + + if (not defined $from) { + # for ascii and UTF-8, no conversion needed to get an utf-8 + # string. + } else { + $text = $from->decode($text, 0); + } + + if (not defined $to) { + # Already in UTF-8, no conversion needed + } else { + $text = $to->encode($text, 0); + } + + return $text; +} + +=back + +=head1 FUTURE DIRECTIONS + +One shortcoming of the current TransTractor is that it can't handle +translated document containing all languages, like debconf templates, or +.desktop files. + +To address this problem, the only interface changes needed are: + +=over 2 + +=item - + +take a hash as po_in_name (a list per language) + +=item - + +add an argument to translate to indicate the target language + +=item - + +make a pushline_all function, which would make pushline of its content for +all language, using a map-like syntax: + + $self->pushline_all({ "Description[".$langcode."]=". + $self->translate($line,$ref,$langcode) + }); + +=back + +Will see if it's enough ;) + +=head1 AUTHORS + + Denis Barbier <barbier@linuxfr.org> + Martin Quinson (mquinson#debian.org) + Jordi Vilalta <jvprat@gmail.com> + +=cut + +1; Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Common.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Common.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Common.pm (revision 2840) @@ -0,0 +1,246 @@ +# Locale::Po4a::Common -- Common parts of the po4a scripts and utils +# +# Copyright 2005 by Jordi Vilalta <jvprat@gmail.com> +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING). +# +# This module has common utilities for the various scripts of po4a + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Common - common parts of the po4a scripts and utils + +=head1 DESCRIPTION + +Locale::Po4a::Common contains common parts of the po4a scripts and some useful +functions used along the other modules. + +In order to use Locale::Po4a programatically, one may want to disable +the use of Text::WrapI18N, by writing e.g. + + use Locale::Po4a::Common qw(nowrapi18n); + use Locale::Po4a::Text; + +instead of: + + use Locale::Po4a::Text; + +Ordering is important here: as most Locale::Po4a modules themselves +load Locale::Po4a::Common, the first time this module is loaded +determines whether Text::WrapI18N is used. + +=cut + +package Locale::Po4a::Common; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Exporter); +@EXPORT = qw(wrap_msg wrap_mod wrap_ref_mod textdomain gettext dgettext); + +use 5.006; +use strict; +use warnings; + +sub import { + my $class=shift; + + my $wrapi18n=1; + if (exists $_[0] && defined $_[0] && $_[0] eq 'nowrapi18n') { + shift; + $wrapi18n=0; + } + $class->export_to_level(1, $class, @_); + + return if defined &wrapi18n; + + if ($wrapi18n && -t STDERR && -t STDOUT && eval { require Text::WrapI18N }) { + + # Don't bother determining the wrap column if we cannot wrap. + my $col=$ENV{COLUMNS}; + if (!defined $col) { + my @term=eval "use Term::ReadKey; Term::ReadKey::GetTerminalSize()"; + $col=$term[0] if (!$@); + # If GetTerminalSize() failed we will fallback to a safe default. + # This can happen if Term::ReadKey is not available + # or this is a terminal-less build or such strange condition. + } + $col=76 if (!defined $col); + + eval ' use Text::WrapI18N qw($columns); + $columns = $col; + '; + + eval ' sub wrapi18n($$$) { Text::WrapI18N::wrap($_[0],$_[1],$_[2]) } ' + } else { + # If we cannot wrap, well, that's too bad. Survive anyway. + eval ' sub wrapi18n($$$) { $_[0].$_[2] } ' + } +} + +sub min($$) { + return $_[0] < $_[1] ? $_[0] : $_[1]; +} + +=head1 FUNCTIONS + +=head2 Showing output messages + +=over + +=item + +show_version($) + +Shows the current version of the script, and a short copyright message. It +takes the name of the script as an argument. + +=cut + +sub show_version { + my $name = shift; + + print sprintf(gettext( + "%s version %s.\n". + "written by Martin Quinson and Denis Barbier.\n\n". + "Copyright (C) 2002, 2003, 2004 Software in the Public Interest, Inc.\n". + "This is free software; see source code for copying\n". + "conditions. There is NO warranty; not even for\n". + "MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." + ), $name, $Locale::Po4a::TransTractor::VERSION)."\n"; +} + +=item + +wrap_msg($@) + +This function displays a message the same way than sprintf() does, but wraps +the result so that they look nice on the terminal. + +=cut + +sub wrap_msg($@) { + my $msg = shift; + my @args = @_; + + return wrapi18n("", "", sprintf($msg, @args))."\n"; +} + +=item + +wrap_mod($$@) + +This function works like wrap_msg(), but it takes a module name as the first +argument, and leaves a space at the left of the message. + +=cut + +sub wrap_mod($$@) { + my ($mod, $msg) = (shift, shift); + my @args = @_; + + $mod .= ": "; + my $spaces = " " x min(length($mod), 15); + return wrapi18n($mod, $spaces, sprintf($msg, @args))."\n"; +} + +=item + +wrap_ref_mod($$$@) + +This function works like wrap_msg(), but it takes a file:line reference as the +first argument, a module name as the second one, and leaves a space at the left +of the message. + +=back + +=cut + +sub wrap_ref_mod($$$@) { + my ($ref, $mod, $msg) = (shift, shift, shift); + my @args = @_; + + if (!$mod) { + # If we don't get a module name, show the message like wrap_mod does + return wrap_mod($ref, $msg, @args); + } else { + $ref .= ": "; + my $spaces = " " x min(length($ref), 15); + $msg = "$ref($mod)\n$msg"; + return wrapi18n("", $spaces, sprintf($msg, @args))."\n"; + } +} + +=head2 Wrappers for other modules + +=over + +=item + +Locale::Gettext + +When the Locale::Gettext module cannot be loaded, this module provide dummy +(empty) implementation of the following functions. In that case, po4a +messages won't get translated but the program will continue to work. + +If Locale::gettext is present, this wrapper also calls +setlocale(LC_MESSAGES, "") so callers don't depend on the POSIX module +either. + +=over + +=item + +bindtextdomain($$) + +=item + +textdomain($) + +=item + +gettext($) + +=item + +dgettext($$) + +=back + +=back + +=cut + +BEGIN { + if (eval { require Locale::gettext }) { + import Locale::gettext; + require POSIX; + POSIX::setlocale(&POSIX::LC_MESSAGES, ''); + } else { + eval ' + sub bindtextdomain($$) { } + sub textdomain($) { } + sub gettext($) { shift } + sub dgettext($$) { return $_[1] } + ' + } +} + +1; +__END__ + +=head1 AUTHORS + + Jordi Vilalta <jvprat@gmail.com> + +=head1 COPYRIGHT AND LICENSE + +Copyright 2005 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Halibut.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Halibut.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Halibut.pm (revision 2840) @@ -0,0 +1,417 @@ +#!/usr/bin/perl -w + +# Copyright (c) 2004-2008 by Nicolas FRANÇOIS <nicolas.francois@centraliens.net> +# +# This file is part of po4a. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with po4a; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Halibut - convert Halibut documents and derivates from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Halibut is a module to help the translation of Halibut documents into +other [human] languages. + +This module contains the definitions of common Halibut commands and +environments. + +=head1 STATUS OF THIS MODULE + +This module is still beta. +Please send feedback and feature requests. + +=head1 CAVEAT + +Some constructs are badly supported. The known ones are documented below. + +=head2 Verbatim blocks + + \c foo + \c bar + +The verbatim block is not considered as a whole. Each line will be +translated separately. + +=head1 SEE ALSO + +L<Locale::Po4a::TeX(3pm)|Locale::Po4a::TeX>, +L<Locale::Po4a::TransTractor(3pm)|Locale::Po4a::TransTractor>, +L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Nicolas François <nicolas.francois@centraliens.net> + +=head1 COPYRIGHT AND LICENSE + +Copyright 2004-2008 by Nicolas FRANÇOIS <nicolas.francois@centraliens.net>. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see COPYING file). + +=cut + +package Locale::Po4a::Halibut; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw($VERSION @ISA @EXPORT); +$VERSION= $Locale::Po4a::TeX::VERSION; +@ISA= qw(Locale::Po4a::TeX); +@EXPORT= qw(); + +use Locale::Po4a::Common; +use Locale::Po4a::TeX; +use subs qw(&parse_definition_file + ®ister_generic_command &is_closed &translate_buffer + ®ister_verbatim_environment + &generic_command + &in_verbatim + &get_leading_command); +*parse_definition_file = \&Locale::Po4a::TeX::parse_definition_file; +*get_leading_command = \&Locale::Po4a::TeX::get_leading_command; +*register_generic_command = \&Locale::Po4a::TeX::register_generic_command; +*register_verbatim_environment = \&Locale::Po4a::TeX::register_verbatim_environment; +*generic_command = \&Locale::Po4a::TeX::generic_command; +*is_closed = \&Locale::Po4a::TeX::is_closed; +*in_verbatim = \&Locale::Po4a::TeX::in_verbatim; +*translate_buffer = \&Locale::Po4a::TeX::translate_buffer; +use vars qw($RE_ESCAPE $ESCAPE + $RE_VERBATIM + $RE_COMMENT $RE_PRE_COMMENT + $no_wrap_environments $separated_commands + %commands %environments + %command_categories %separated + %env_separators %debug + %translate_buffer_env + @exclude_include @comments); +*RE_ESCAPE = \$Locale::Po4a::TeX::RE_ESCAPE; +*ESCAPE = \$Locale::Po4a::TeX::ESCAPE; +*RE_VERBATIM = \$Locale::Po4a::TeX::RE_VERBATIM; +*RE_COMMENT = \$Locale::Po4a::TeX::RE_COMMENT; +*RE_PRE_COMMENT = \$Locale::Po4a::TeX::RE_PRE_COMMENT; +*no_wrap_environments = \$Locale::Po4a::TeX::no_wrap_environments; +*separated_commands = \$Locale::Po4a::TeX::separated_commands; +*commands = \%Locale::Po4a::TeX::commands; +*environments = \%Locale::Po4a::TeX::environments; +*command_categories = \%Locale::Po4a::TeX::command_categories; +*separated = \%Locale::Po4a::TeX::separated; +*env_separators = \%Locale::Po4a::TeX::env_separators; +*debug = \%Locale::Po4a::TeX::debug; +*translate_buffer_env = \%Locale::Po4a::TeX::translate_buffer_env; +*exclude_include = \@Locale::Po4a::TeX::exclude_include; +*comments = \@Locale::Po4a::TeX::comments; + +#$ESCAPE = "\\"; +#$RE_ESCAPE = "\\\\"; +#$RE_VERBATIM = "\@example"; +$RE_VERBATIM = "PO4A_FAKE_VERBATIM"; +#$RE_COMMENT = "\\\@(?:c|comment)\\b"; +$RE_COMMENT = "PO4A_FAKE_COMMENT"; + +sub docheader { + return "\\# This file was generated with po4a. Translate the source file.\n". + "\n"; +} + +my %break_line = (); + +# translate_line_command indicate if the arguments to the command handled +# by line_command() should be translated: +# undefined: arguments are not translated +# 0: there should be no arguments +# 1: arguments should be translated +my %translate_line_command = (); + +sub parse { + my $self = shift; + my ($line,$ref); + my $paragraph = ""; # Buffer where we put the paragraph while building + my @env = (); # environment stack + my $t = ""; +# $docheader_pushed = 0; + + LINE: + undef $self->{type}; + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { + chomp($line); + $self->{ref}="$ref"; + + if ($line =~ /^\s*\\\s*po4a\s*:/) { + parse_definition_line($self, $line); + goto LINE; + } + + my $t; + ($paragraph, $t, @env) = parse_line($self, $line, $paragraph, \@env); + $self->pushline($t); + + + # Reinit the loop + ($line,$ref)=$self->shiftline(); + undef $self->{type}; + } + + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + } +} # end of parse + +sub parse_line { + my $self = shift; + my $line = shift; + my $paragraph = shift; + my $env = shift; + my @e = @$env; + my $translated = ""; + + my $closed = 1; + if (!in_verbatim(@e)) { + $closed = is_closed($paragraph); + } +# if (not $closed) { +# print "not closed. line: '$line'\n para: '$paragraph'\n"; +# } + +#warn "closed'$closed'$line'$paragraph'\n"; + if ($closed and $line =~ /^\s*$/) { + # An empty line. This indicates the end of the current + # paragraph. + $paragraph .= $line."\n"; + if (length($paragraph)) { + ($translated, @e) = translate_buffer($self,$paragraph,undef,@e); + $paragraph=""; + } + } elsif ($line =~ m/^\\input /) { + if (length($paragraph)) { + ($translated, @e) = translate_buffer($self,$paragraph,undef,@e); + $paragraph=""; + } + $translated .= $line."\n"; + } elsif ($line =~ m/^$RE_COMMENT/) { + $translated = $line."\n"; + } elsif ( $closed + and (is_closed($line) or $line =~ /^\\[ce] /) + and ($line =~ /^\\([^ ]*?)( +.*)?$/)) { + my ($command,$variant,$args,$buffer); + if ($break_line{$1}) { + my @a = (); + $variant = ""; + $args = \@a; + $command = $1; + $buffer = $2||""; + } else { + ($command,$variant,$args,$buffer) = get_leading_command($self, $line); + } + if ( $break_line{$command} + and not ( ($command eq "c" or $command eq "e") + and defined $args->[0])) { +# NOTE: This is just a workaround: "\c " is a verbatim line +# and \c{...} is just a verbatim block + my $t; + if (length($paragraph)) { + ($t, @e) = translate_buffer($self,$paragraph,undef,@e); + $translated .= $t; + $paragraph=""; + } + ($t, @e) = generic_command($self, $command, $variant, $args, \@e); + $translated .= $t; + + my $arg = $buffer; + my @args = (); + if (defined $arg and length $arg) { + # FIXME: keep the spaces ? + $arg =~ s/\s*$//s; + @args= (" ", $arg); + } + ($t, @e) = line_command($self, $command, "", \@args, \@e, 1); + $translated .= $t."\n"; + } else { + # continue the same paragraph + $paragraph .= $line."\n"; + } + } else { + # continue the same paragraph + $paragraph .= $line."\n"; + } + + return ($paragraph, $translated, @e); +} + +sub line_command { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "line_command($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'}); + + my $translated = ""; # $ESCAPE.$command; + my $line = $args->[1]; +#warn "line_command: '$line'\n"; + if (defined $line and length $line) { + if ( defined $translate_line_command{$command} + and $translate_line_command{$command}) { + # $no_wrap could be forced to 1, but it should already be set + $no_wrap = 1; + $line =~ s/^(\s*)//; + my $spaces = $1 || ""; + my ($t,$e) = $self->translate_buffer($line,$no_wrap,@$env,$command); +#warn "line_command: '$t'\n"; + $translated .= $spaces.$t; + } else { + $translated .= $line; + } + } + print "($translated,@$env)\n" + if ($debug{'commands'}); + return ($translated,@$env); +} + + +# 3.2 Simple inline formatting commands +# 3.2.1 `\e': Emphasising text +# inline. extract only if alone +register_generic_command("-e,{_}"); +$translate_line_command{e} = 1; +$break_line{e} = 1; +# 3.2.2 `\c' and `\cw': Displaying computer code inline +# inline. extract only if alone +# NOTE: \c and \c{...} differs. +# \c is marked as a break_line command, but this is reversed in +# parse_line when the \c{...} form is used. +register_generic_command("-c,{_}"); +$translate_line_command{c} = 1; +$break_line{c} = 1; +register_generic_command("-cw,{_}"); +# 3.2.3 `\q': Quotation marks +# inline. extract only if alone +register_generic_command("-q,{_}"); +# 3.2.4 `\-' and `\_': Non-breaking hyphens and spaces +# inline. + +# 3.2.5 `\date': Automatic date generation +# inline. + +# 3.2.6 `\W': WWW hyperlinks +# inline. extract only if alone +register_generic_command("-W,{_}"); +# 3.2.7 `\u': Specifying arbitrary Unicode characters +# inline. + +# 3.2.8 `\k' and `\K': Cross-references to other sections +# inline. They should not be translated. extract only if alone +# FIXME: it will expand to "Section ..." or "section ..." +# Section and section should be translated. +register_generic_command("-k,{}"); +register_generic_command("-K,{}"); +# 3.2.9 `\#': Inline comments +# inline. But can be removed from the head or tail. +register_generic_command("-#,{}"); +$translate_line_command{"#"} = 0; +$break_line{"#"} = 1; +# 3.3 Paragraph-level commands +# 3.3.1 `\c': Displaying whole paragraphs of computer code +# see above +# 3.3.2 `\b', `\n', `\dt', `\dd', `\lcont': Lists +register_generic_command("*b,"); +register_generic_command("*n,"); # FIXME: \n{this-one} not supported? +register_generic_command("*dd,"); +register_generic_command("*dt,"); +# 3.3.2.4 Continuing list items into further paragraphs +register_generic_command("*lcont,{_}"); # registered, but redefined +$commands{lcont} = sub { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + my ($t,@e)=("",@$env); + my $translated = $ESCAPE.$command.$variant."{"; + my $text = $args->[1]; + my $paragraph = ""; + while ( $text =~ s/^(.*?)\n(.*)$/$2/s + or $text =~ s/^([^\n]+)$//s) { + ($paragraph, $t, @e) = parse_line($self, $1, $paragraph, \@e); + $translated .= $t; + } + ($t, @e) = translate_buffer ($self, $paragraph, $no_wrap, @e); + $translated .= $t; + $translated .= "}"; + + return ($translated, @$env); +}; +# 3.3.3 `\rule': Horizontal rules +register_generic_command("rule,"); # TODO: TBC does it break paragraphs +# 3.3.4 `\quote': Indenting multiple paragraphs as a long quotation +register_generic_command("*quote,{_}"); # TODO: TBC +# 3.3.5 `\C', `\H', `\S', `\A', `\U': Chapter and section headings +# FIXME: What happens if the the line is rewrapped? +# NOTE: The name of the section is not translated. +register_generic_command("*C,{}"); +register_generic_command("*S0,{}"); # Synonym for \H +register_generic_command("*H,{}"); +register_generic_command("*S,{}"); +register_generic_command("*S1,{}"); # Synonym for \S +register_generic_command("*S2,{}"); +register_generic_command("*S3,{}"); # FIXME: and so on +# FIXME: \S{question-about-fish}{Question} +register_generic_command("*A,{}"); +register_generic_command("*U,{}"); +# 3.3.6 `\copyright', `\title', `\versionid': Miscellaneous blurb commands +register_generic_command("*title,"); +register_generic_command("*copyright,"); +register_generic_command("*versionid,"); +# 3.4 Creating a bibliography +# nocite +register_generic_command("*nocite,{}"); +# B +register_generic_command("*B,{}"); +# BR +register_generic_command("*BR,{}"); # FIXME: \BR{freds-book} [Fred1993] +# 3.5 Creating an index +# 3.5.1 Simple indexing +# \i: inline \i{index} or \i\x{grep} +# \ii +register_generic_command("-ii,{_}"); +# \IM: inline. Variable number of arguments +register_generic_command("*IM,{_}"); +$translate_line_command{IM} = 1; +$break_line{IM} = 1; +# 3.6 Configuring Halibut +# \cfg +register_generic_command("+cfg,{}{_}"); # NOTE: the new command is not registered +# 3.7 Defining macros +register_generic_command("*define,{}"); # FIXME: line +$translate_line_command{define} = 1; +$break_line{define} = 1; + +1; Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/KernelHelp.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/KernelHelp.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/KernelHelp.pm (revision 2840) @@ -0,0 +1,175 @@ +# Locale::Po4a::KernelHelp -- Convert kernel configuration help from/to PO files +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING). +# +# This module converts POD to PO file, so that it becomes possible to +# translate POD formatted documentation. See gettext documentation for +# more info about PO files. + +############################################################################ +# Modules and declarations +############################################################################ + +use Pod::Parser; +use Locale::Po4a::TransTractor qw(process new); +use Locale::Po4a::Common; + +package Locale::Po4a::KernelHelp; + +use 5.006; +use strict; +use warnings; + +require Exporter; + +use vars qw(@ISA @EXPORT $AUTOLOAD); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); # new process write read writepo readpo); + +my $debug=0; + +sub initialize {} + +sub parse { + my $self=shift; + my ($line,$ref); + my $paragraph=""; # Buffer where we put the paragraph while building + my ($status)=0; # Syntax of KH is: + # description<nl>variable<nl>help text<nl><nl> + # Status will be: + # 0 1 2 3 0 + + my ($desc,$variable); + + LINE: + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { + chomp($line); + print STDERR "status=$status;Seen >>$line<<:" if $debug; + + if ($line =~ /^\#/) { + print STDERR "comment.\n" if $debug; + $self->pushline("$line\n"); + } elsif ($status == 0) { + if ($line =~ /\S/) { + print STDERR "short desc.\n" if $debug; + $desc=$line; + $status ++; + } else { + print STDERR "empty line.\n" if $debug; + $self->pushline("$line\n"); + } + } elsif ($status == 1) { + print STDERR "var name.\n" if $debug; + $variable=$line; + $status++; + + $self->pushline($self->translate($desc,$ref,"desc_$variable"). + "\n$variable\n"); + + } elsif ($status == 2) { + $line =~ s/^ //; + if ($line =~ /\S/) { + print STDERR "paragraph line.\n" if $debug; + $paragraph .= $line."\n"; + } else { + print STDERR "end of paragraph.\n" if $debug; + $status++; + $paragraph=$self->translate($paragraph, + $ref, + "helptxt_$variable"); + $paragraph =~ s/^/ /gm; + $self->pushline("$paragraph\n"); + $paragraph =""; + } + } elsif ($status == 3) { + if ($line =~ s/^ //) { + if ($line =~ /\S/) { + print "begin of paragraph.\n" if $debug; + $paragraph = $line."\n"; + $status--; + } else { + print "end of config option.\n" if $debug; + $status=0; + $self->pushline("\n"); + } + } else { + $self->unshiftline($line,$ref); + $status=0; + } + } else { + die wrap_ref_mod($ref, "po4a::kernelhelp", gettext("Syntax error")); + } + + # Reinit the loop + ($line,$ref)=$self->shiftline(); + } +} + +sub docheader { + return <<EOT; +# +# ***************************************************** +# * GENERATED FILE, DO NOT EDIT * +# * THIS IS NO SOURCE FILE, BUT RESULT OF COMPILATION * +# ***************************************************** +# +# This file was generated by po4a(7). Do not store it (in VCS, for example), +# but store the PO file used as source file by pod-translate. +# +# In fact, consider this as a binary, and the PO file as a regular .c file: +# If the PO get lost, keeping this translation up-to-date will be harder. +# +EOT +} +1; + +############################################################################## +# Module return value and documentation +############################################################################## + +1; +__END__ + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::KernelHelp - convert kernel configuration help from/to PO files + +=head1 DESCRIPTION + +Locale::Po4a::KernelHelp is a module to help the translation of +documentation for the Linux kernel configuration options into other [human] +languages. + +=head1 STATUS OF THIS MODULE + +This module is just written, and needs more tests. Most of the needed work +will concern the tools used to parse this file (and configure the kernel), +so that they accept to read the documentation from another (translated) +file. + +=head1 SEE ALSO + +L<Pod::Parser>, +L<Locale::Po4a::Man(3pm)>, +L<Locale::Po4a::Pod(3pm)>, +L<Locale::Po4a::TransTractor(3pm)>, +L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Denis Barbier <barbier@linuxfr.org> + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Xhtml.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Xhtml.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Xhtml.pm (revision 2840) @@ -0,0 +1,237 @@ +#!/usr/bin/perl + +# Po4a::Xhtml.pm +# +# extract and translate translatable strings from XHTML documents. +# +# This code extracts plain text from tags and attributes from strict XHTML +# documents. +# +# Copyright (c) 2005 by Yves Rütschlé <po4a@rutschle.net> +# Copyright (c) 2007-2008 by Nicolas François <nicolas.francois@centraliens.net> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Xhtml - convert XHTML documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Xhtml is a module to help the translation of XHTML documents into +other [human] languages. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +These are this module's particular options: + +=over 4 + +=item B<includessi>[B<=>I<rootpath>] + +Include files specified by an include SSI (Server Side Includes) element +(e.g. <!--#include virtual="/foo/bar.html" -->). + +B<Note:> You should use it only for static files. + +An additional I<rootpath> parameter can be specified. It specifies the root +path to find files included by a B<virtual> attribute. + +=back + +=head1 STATUS OF THIS MODULE + +This module is fully functional, as it relies in the L<Locale::Po4a::Xml> +module. This only defines the translatable tags and attributes. + +"It works for me", which means I use it successfully on my personal Web site. +However, YMMV: please let me know if something doesn't work for you. In +particular, tables are getting no testing whatsoever, as we don't use them. + +=head1 SEE ALSO + +L<Locale::Po4a::TransTractor(3pm)>, L<Locale::Po4a::Xml(3pm)>, L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Yves Rütschlé <po4a@rutschle.net> + Nicolas François <nicolas.francois@centraliens.net> + +=head1 COPYRIGHT AND LICENSE + + Copyright (c) 2004 by Yves Rütschlé <po4a@rutschle.net> + Copyright (c) 2007-2008 by Nicolas François <nicolas.francois@centraliens.net> + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +package Locale::Po4a::Xhtml; + +use 5.006; +use strict; +use warnings; + +use Locale::Po4a::Xml; +use vars qw(@tag_types); +*tag_types = \@Locale::Po4a::Xml::tag_types; + +use Locale::Po4a::Common; +use Carp qw(croak); + +use vars qw(@ISA); +@ISA = qw(Locale::Po4a::Xml); + +sub tag_extract_SSI { + my ($self,$remove)=(shift,shift); + my ($eof,@tag)=$self->get_string_until("-->", + {include=>1, + remove=>$remove, + unquoted=>1}); + my ($t,$r) = @tag; + if ($t =~ m/<!--#include (file|virtual)="(.*?)"\s-->/s) { + my $includefile; + if ($1 eq "file") { + $includefile = "."; + } else { + $includefile = $self->{options}{'includessi'}; + } + $includefile .= $2; + if (!$remove) { + $self->get_string_until("-->", + {include=>1, + remove=>1, + unquoted=>1}); + } + my $linenum=0; + my @include; + + open (my $in, $includefile) + or croak wrap_mod("po4a::xml", + dgettext("po4a", "Can't read from %s: %s"), + $includefile, $!); + while (defined (my $includeline = <$in>)) { + $linenum++; + my $includeref=$includefile.":$linenum"; + push @include, ($includeline,$includeref); + } + close $in + or croak wrap_mod("po4a::xml", + dgettext("po4a", "Can't close %s after reading: %s"), + $includefile, $!); + + while (@include) { + my ($ir, $il) = (pop @include, pop @include); + $self->unshiftline($il,$ir); + } + $t =~ s/<!--#include/<!-- SSI included by po4a: /; + $self->unshiftline($t, $r); + } + return ($eof,@tag); +} + +sub initialize { + my $self = shift; + my %options = @_; + + $self->{options}{'includessi'}=''; + + $self->SUPER::initialize(%options); + + $self->{options}{'wrap'}=1; + $self->{options}{'doctype'}=$self->{options}{'doctype'} || 'html'; + + # Default tags are translated (text rewrapped), and introduce a + # break. + # The following list indicates the list of tags which should be + # translated without rewrapping. + $self->{options}{'_default_translated'}.=' + W<pre> + '; + + # The following list indicates the list of tags which should be + # translated inside the current block, whithout introducing a + # break. + $self->{options}{'_default_inline'}.=' + <a> + <abbr> + <acronym> + <b> + <big> + <bdo> + <button> + <cite> + <code> + <del> + <dfn> + <em> + <i> + <ins> + <input> + <kbd> + <label> + <object> + <q> + <samp> + <select> + <small> + <span> + <strong> + <sub> + <sup> + <textarea> + <tt> + <u> + <var> + '; + + # Ignored tags: <img> + # Technically, <img> is an inline tag, but setting it as such is + # annoying, and not usually useful, unless you use images to + # write text (in which case you have bigger problems than this + # program not inlining img: you now have to translate all your + # images. That'll teach you). + # If you choose to translate images, you may also want to set + # <map> as placeholder and <area> as inline. + + $self->{options}{'_default_attributes'}.=' + alt + lang + title + '; + $self->treat_options; + + if ( defined $self->{options}{'includessi'} + and length $self->{options}{'includessi'}) { + foreach (@tag_types) { + if ($_->{beginning} eq "!--#") { + $_->{f_extract} = \&tag_extract_SSI; + } + } + # FIXME: the directory may be named "1" ;( + if ($self->{options}{'includessi'} eq "1") { + $self->{options}{'includessi'} = "."; + } + } +} Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Guide.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Guide.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Guide.pm (revision 2840) @@ -0,0 +1,151 @@ +#!/usr/bin/perl + +# Po4a::Guide.pm +# +# extract and translate translatable strings from Guide XML documents. +# +# This code extracts plain text from tags and attributes on Guide XML +# documents. +# +# Copyright (c) 2004 by Jordi Vilalta <jvprat@gmail.com> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Guide - convert Guide XML documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Guide is a module to help in the translation of the Gentoo +Linux documentation in the Guide XML format into other [human] languages. + +This format is documented here: http://www.gentoo.org/doc/en/xml-guide.xml + +=head1 STATUS OF THIS MODULE + +This module is fully functional, as it relies in the L<Locale::Po4a::Xml> +module. This only defines the translatable tags and attributes. + +The only known issue is that it doesn't include files with the <include +href="..."> tag, but you can translate all those files alone, and it's usually +better to have them separated. + +=head1 SEE ALSO + +L<Locale::Po4a::TransTractor(3pm)>, L<Locale::Po4a::Xml(3pm)>, L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Jordi Vilalta <jvprat@gmail.com> + +=head1 COPYRIGHT AND LICENSE + +Copyright (c) 2004 by Jordi Vilalta <jvprat@gmail.com> + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +package Locale::Po4a::Guide; + +use 5.006; +use strict; +use warnings; + +use Locale::Po4a::Xml; + +use vars qw(@ISA); +@ISA = qw(Locale::Po4a::Xml); + +sub initialize { + my $self = shift; + my %options = @_; + +#TODO: <include href="..."> includes a file + $self->SUPER::initialize(%options); + $self->{options}{'_default_translated'}.=' + w<abstract> + <author> + <b> + <brite> + <c> + <codenote> + <comment> + <const> + <date> + w<dd> + w<dt> + <e> + <i> + <ident> + w<impo> + <keyword> + w<li> + <mail> + w<note> + w<p> + <path> + W<pre> + <stmt> + <sub> + w<subtitle> + w<summary> + <sup> + w<th> + w<ti> + w<title> + <uri> + <var> + <version> + w<warn>'; + $self->{options}{'_default_attributes'}.=' + <author>title + <figure>caption + <figure>link + <figure>short + <guide>lang + <guide>link + <p>by + <pre>caption'; + $self->{options}{'_default_inline'}.=' + <b> + <brite> + <c> + <const> + <e> + <i> + <ident> + <img> + <keyword> + <mail> + <path> + <stmt> + <sub> + <sup> + <uri> + <var>'; + $self->treat_options; +} Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Dia.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Dia.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Dia.pm (revision 2840) @@ -0,0 +1,113 @@ +#!/usr/bin/perl + +# Po4a::Dia.pm +# +# extract and translate translatable strings from Dia diagrams. +# +# This code extracts plain text from string tags on uncompressed Dia +# diagrams. +# +# Copyright (c) 2004 by Jordi Vilalta <jvprat@gmail.com> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Dia - convert uncompressed Dia diagrams from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Dia is a module to help the translation of diagrams in the +uncompressed Dia format into other [human] languages. + +You can get Dia (the graphical editor for these diagrams) from: + http://www.gnome.org/projects/dia/ + +=head1 TRANSLATING WITH PO4A::DIA + +This module only translates uncompressed Dia diagrams. You can save your +uncompressed diagrams with Dia itself, unchecking the "Compress diagram +files" at the "Save Diagram" dialog. + +Another way is to uncompress the dia files from command line with: + gunzip < original.dia > uncompressed.dia + +=head1 STATUS OF THIS MODULE + +This module is fully functional, as it relies in the L<Locale::Po4a::Xml> +module. This only defines the translatable tags (E<lt>dia:stringE<gt>), and +filters the internal strings (the content of the E<lt>dia:diagramdataE<gt> +tag), not interesting for translation. + +=head1 SEE ALSO + +L<Locale::Po4a::TransTractor(3pm)>, L<Locale::Po4a::Xml(3pm)>, L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Jordi Vilalta <jvprat@gmail.com> + +=head1 COPYRIGHT AND LICENSE + +Copyright (c) 2004 by Jordi Vilalta <jvprat@gmail.com> + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +package Locale::Po4a::Dia; + +use 5.006; +use strict; +use warnings; + +use Locale::Po4a::Xml; + +use vars qw(@ISA); +@ISA = qw(Locale::Po4a::Xml); + +sub initialize { + my $self = shift; + my %options = @_; + + $self->SUPER::initialize(%options); + $self->{options}{'nostrip'}=1; + $self->{options}{'_default_translated'}.=' <dia:string>'; + $self->treat_options; +} + +sub found_string { + my ($self,$text,$ref,$options)=@_; + return $text if $text =~ m/^\s*$/s; + + #We skip the paper type string + if ( $self->get_path() !~ /<dia:diagramdata>/ ) { + $text =~ /^#(.*)#$/s; + $text = "#".$self->translate($1,$ref,"String", + 'wrap'=>$self->{options}{'wrap'})."#"; + } + + return $text; +} Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Wml.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Wml.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Wml.pm (revision 2840) @@ -0,0 +1,172 @@ +#!/usr/bin/perl -w + +# Po4a::Wml.pm +# +# extract and translate translatable strings from a WML (web markup language) documents +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Wml - convert WML (web markup language) documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Wml is a module to help the translation of WML documents into +other [human] languages. Do not mixup the WML we are speaking about here +(web markup language) and the WAP crap used on cell phones. + +Please note that this module relies upon the Locale::Po4a::Xhtml +module, which also relies upon the Locale::Po4a::Xml module. This +means that all tags for web page expressions are assumed to be written +in the XHTML syntax. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +NONE. + +=head1 STATUS OF THIS MODULE + +This module works for some simple documents, but is still young. +Currently, the biggest issue of the module is probably that it cannot +handle documents that contain non-XML inline tags such as <email +"foo@example.org">, which are often defined in the WML. Improvements +will be added in the future releases. + +=cut + +package Locale::Po4a::Wml; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::Xhtml); +@EXPORT = qw(); + +use Locale::Po4a::Xhtml; +use File::Temp; + +sub initialize { + my $self = shift; + my %options = @_; + + $self->SUPER::initialize(%options); + + $self->treat_options; +} + +sub read { + my ($self,$filename)=@_; + my $tmp_filename; + (undef,$tmp_filename)=File::Temp->tempfile("po4aXXXX", + DIR => "/tmp", + SUFFIX => ".xml", + OPEN => 0, + UNLINK => 0) + or die wrap_msg(gettext("Can't create a temporary XML file: %s"), $!); + my $file; + open FILEIN,"$filename" or die "Cannot read $filename: $!\n"; + { + $/ = undef; + $file=<FILEIN>; + } + $/ = "\n"; + + # Mask perl cruft out of XML sight + while ( ($file =~ m|^(.*?)<perl>(.*?)</perl>(.*?)$|ms) + or ($file =~ m|^(.*?)<:(.*?):>(.*)$|ms)) { + my ($pre,$in,$post) = ($1,$2,$3); + $in =~ s/</PO4ALT/g; + $in =~ s/>/PO4AGT/g; + $file = "${pre}<!--PO4ABEGINPERL${in}PO4AENDPERL-->$post"; + } + + # Mask mp4h cruft + while ($file =~ s|^#(.*)$|<!--PO4ASHARPBEGIN$1PO4ASHARPEND-->|m) { + my $line = $1; + print STDERR "PROTECT HEADER: $line\n" + if $self->debug(); + if ($line =~ m/title="([^"]*)"/) { #) {#"){ + warn "FIXME: We should translate the page title: $1\n"; + } + } + + # Validate define-tag tag's argument + $file =~ s|(<define-tag\s+)([^\s>]+)|$1PO4ADUMMYATTR="$2"|g; + + # Flush the result to disk + open OUTFILE,">$tmp_filename"; + print OUTFILE $file; + close INFILE; + close OUTFILE or die "Cannot write $tmp_filename: $!\n"; + + push @{$self->{DOCXML}{infile}}, $tmp_filename; + $self->{DOCWML}{$tmp_filename} = $filename; + $self->Locale::Po4a::TransTractor::read($tmp_filename); + unlink "$tmp_filename"; +} + +sub parse { + my $self = shift; + + foreach my $filename (@{$self->{DOCXML}{infile}}) { + $self->Locale::Po4a::Xml::parse_file($filename); + my $org_filename = $self->{DOCWML}{$filename}; + + # Fix the references + foreach my $msgid (keys %{$self->{TT}{po_out}{po}}) { + $self->{TT}{po_out}{po}{$msgid}{'reference'} =~ + s|$filename(:\d+)|$org_filename$1|o; + } + + # Get the document back (undoing our WML masking) + # FIXME: need to join the file first, and then split? + my @doc_out; + foreach my $line (@{$self->{TT}{doc_out}}) { + $line =~ s/^<!--PO4ASHARPBEGIN(.*?)PO4ASHARPEND-->/#$1/mg; + $line =~ s/<!--PO4ABEGINPERL(.*?)PO4AENDPERL-->/<:$1:>/sg; + $line =~ s/(<define-tag\s+)PO4ADUMMYATTR="([^"]*)"/$1$2/g; + $line =~ s/PO4ALT/</sg; + $line =~ s/PO4AGT/>/sg; + push @doc_out, $line; + } + $self->{TT}{doc_out} = \@doc_out; + } +} + +1; + +=head1 AUTHORS + + Martin Quinson (mquinson#debian.org) + Noriada Kobayashi <nori1@dolphin.c.u-tokyo.ac.jp> + +=head1 COPYRIGHT AND LICENSE + + Copyright 2005 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Debconf.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Debconf.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Debconf.pm (revision 2840) @@ -0,0 +1,221 @@ +#!/usr/bin/perl -w + +# Po4a::Debconf.pm +# +# extract and translate translatable strings from debconf templates +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Debconf - convert debconf templates from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Debconf is a module to help the translation of the debconf +templates into other [human] languages. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +NONE. + +=head1 STATUS OF THIS MODULE + +Not tested. + + +DO NOT USE THIS MODULE TO PRODUCE TEMPLATES. It's only good to extract data. + + +=cut + +# Note that the following works. It may help to write a multi-translate + +# sub toto { +# do shift; +# } +# toto({print "ok"}); + + +package Locale::Po4a::Debconf; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + + +sub initialize {} + +sub parse { + my $self = shift; + + my ($line,$lref); + + my ($field, $value, $extended,$ref,$type)=('', '', '', '',''); + my $verb = 0; # whether we are in verbatim mode + + my $escape = sub { + my $str=shift; + $str =~ s/"/\\"/g; + return $str; + }; + + # function in charge of pushing the accumulated material to output + my $handle_field = sub { + my $field=shift; + my $value=shift; + my $extended=shift; + my $ref = shift; + my $type = shift; + + $field =~ s/^(_*)(.*)$/$2/; + my $undercount = length($1) || 0; # number of _ leading the field name + + # Only one leading _: regular translated field + if ($undercount == 1) { + + # the untranslated field + $self->pushline("$field: $value"); + map {$self->pushline(' '.($_||'.'))} split (/\n/,$extended); + + + my $eval='$self->pushline("'.$field.'[FIXME:LANGCODE.ENCODING]: "'; # what to multi-eval + $eval .= '.$self->translate("'.$escape->($value)."\",\"$ref\",\"$type/$field\",wrap=>1)".'."\n".'."\n"; + + my $count = 0; + foreach my $para (split(/\n\n/, $extended)) { + my $wrap = 1; + if ($para =~ /(^|\n)\s/m) { + $wrap = 0; + } + $eval .= ($count?'.':''); + $count ++; + $eval .= '$self->translate("'.$escape->($para)."\",\"$ref\",\"$type/$field\[$count\]\",wrap=>$wrap)"."\n"; + } + + $eval .= ")\n"; + print STDERR $eval if $self->debug(); + eval $eval; + print STDERR "XXXXXXXXXXXXXXXXX\n" if $self->debug(); + + + # two leading _: split on coma and multi-translate each part. No extended value. + } elsif ($undercount == 2) { + $self->pushline("$field: $value"); # the untranslated field + + my $eval='$self->pushline("'.$field.'FIXME[LANGCODE]: "'; # what to multi-eval + + my $first = 1; + for my $part (split(/(?<!\\), */, $value, 0)) + { + $part =~ s/\\,/,/g; + $eval .= ($first?'':'.", "').'.$self->translate("'.$escape->($part)."\",\"$ref\",\"$type/$field chunk\",wrap=>1)"; + $first = 0; + } + $eval .= ")\n"; + + print $eval if $self->debug(); + eval $eval; + + # no leading _: don't touch it + } else { + $self->pushline("$field: $value"); + map {$self->pushline(' '.($_||'.'))} split (/\n/,$extended); + } + }; + + # main loop + ($line,$lref)=$self->shiftline(); + + while (defined($line)) { + # a new field (within a stanza) + if ($line=~/^([-_.A-Za-z0-9]*):\s?(.*)/) { + + $handle_field->($field, $value, $extended, $ref,$type); # deal with previously accumulated + ($field, $value, $extended,$verb)=('', '', '', 0); + + $field=$1; + $value=$2; + $value=~s/\s*$//; + $extended=''; + $ref=$lref; + + $type = $value if $field eq 'Type'; + + die wrap_mod("po4a::debconf", dgettext("po4a", "Translated field in master document: %s"), $field) + if $field =~ m/-/; + + # paragraph separator within extended value + } elsif ($line=~/^\s\.$/) { + $extended.="\n\n"; + + # continuation of extended value + } elsif ($line=~/^\s(.*)/) { + + my $bit=$1; + $verb = 1 if ($bit =~ m/^\s/); + + $bit=~s/\s*$//; + + $extended .= ($verb ? "\n" : ' ') if length $extended && $extended !~ /[\n ]$/; + $extended .= $bit.($verb ? "\n" : ""); + + # this may be an empty line closing the stanza, a comment or even a parse error (if file not DebConf-clean). + } else { + + $handle_field->($field, $value, $extended, $ref,$type); + ($field, $value, $extended,$verb)=('', '', '', 0); + + $self->pushline($line); + + } + + ($line,$lref)=$self->shiftline(); + } + + $handle_field->($field, $value, $extended, $ref,$type); +} + +1; + +=head1 AUTHORS + +This module is loosely inspired from both po-debconf and debconf code. The +adaptation for po4a was done by: + + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + + Copyright 2005 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/TeX.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/TeX.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/TeX.pm (revision 2840) @@ -0,0 +1,1729 @@ +#!/usr/bin/perl -w + +# Copyright (c) 2004, 2005 by Nicolas FRANÇOIS <nicolas.francois@centraliens.net> +# +# This file is part of po4a. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with po4a; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::TeX - convert TeX documents and derivates from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::TeX is a module to help the translation of TeX documents into +other [human] languages. It can also be used as a base to build modules for +TeX-based documents. + +Users should probably use the LaTeX module, which inherite from the TeX module +and contains the definitions of common LaTeX commands. + +=head1 TRANSLATING WITH PO4A::TEX + +This module can be used directly to handle generic TeX documents. +This will split your document in smaller blocks (paragraphs, verbatim +blocks, or even smaller like titles or indexes). + +There are some options (described in the next section) that can customize +this behavior. If this doesn't fit to your document format you're encouraged +to write your own module derived from this, to describe your format's details. +See the section B<WRITING DERIVATE MODULES> below, for the process description. + +This module can also be customized by lines starting with "% po4a:" in the +TeX file. +These customizations are described in the B<INLINE CUSTOMIZATION> section. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +These are this module's particular options: + +=over 4 + +=cut + +package Locale::Po4a::TeX; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(%commands %environments + $RE_ESCAPE $ESCAPE $RE_VERBATIM + $no_wrap_environments + $verbatim_environments + %separated_command + %separated_environment + %translate_buffer_env + &generic_command + ®ister_generic_command + ®ister_generic_environment); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; +use File::Basename qw(dirname); +use Carp qw(croak); + +use Encode; +use Encode::Guess; + +# hash of known commands and environments, with parsing sub. +# See end of this file +use vars qw(%commands %environments); +# hash to describe the number of parameters and which one have to be +# translated. Used by generic commands +our %command_parameters = (); +our %environment_parameters = (); +# hash to describe the separators of environments. +our %env_separators =(); + +# The escape character used to introduce commands. +our $RE_ESCAPE = "\\\\"; +our $ESCAPE = "\\"; +# match the beginning of a verbatim block +our $RE_VERBATIM = "\\\\begin\\{(?:verbatim)\\*?\\}"; +# match the beginning of a comment. +# NOTE: It must contain a group, with chars preceding the comment +our $RE_PRE_COMMENT= "(?<!\\\\)(?:\\\\\\\\)*"; +our $RE_COMMENT= "\\\%"; + +# Space separated list of environments that should not be re-wrapped. +our $no_wrap_environments = "verbatim"; +our $verbatim_environments = "verbatim"; +# hash with the commands that have to be separated (or have to be joined). +# 3 modes are currently used: +# '*' The command is separated if it appear at an extremity of a +# paragraph +# '+' The command is separated, but its arguments are joined together +# with the command name for the translation +# '-' The command is not separated, unless it appear alone on a paragraph +# (e.g. \strong) +our %separated_command = (); +our %separated_environment = (); + +=item B<debug> + +Activate debugging for some internal mechanisms of this module. +Use the source to see which parts can be debugged. + +=item B<no_wrap> + +Comma-separated list of environments which should not be re-wrapped. + +Note that there is a difference between verbatim and no_wrap environments. +There is no command and comments analysis in verbatim blocks. + +If this environment was not already registered, po4a will consider that +this environment does not take any parameters. + +=item B<exclude_include> + +Colon-separated list of files that should not be included by \input and +\include. + +=item B<definitions> + +The name of a file containing definitions for po4a, as defined in the +B<INLINE CUSTOMIZATION> section. +You can use this option if it is not possible to put the definitions in +the document being translated. + +=item B<verbatim> + +Comma-separated list of environments which should be taken as verbatim. + +If this environment was not already registered, po4a will consider that +this environment does not take any parameters. + +=back + +Using these options permits to override the behaviour of the commands defined +in the default lists. + +=head1 INLINE CUSTOMIZATION + +The TeX module can be customized with lines starting by B<% po4a:>. +These lines are interpreted as commands to the parser. +The following commands are recognized: + +=over 4 + +=item B<% po4a: command> I<command1> B<alias> I<command2> + +Indicates that the arguments of the I<command1> command should be +treated as the arguments of the I<command2> command. + +=item B<% po4a: command> I<command1> I<parameters> + +This permit to describe in detail the parameters of the I<command1> +command. +This information will be used to check the number of arguments and their +types. + +You can precede the I<command1> command by + +=over 4 + +=item an asterisk (B<*>) + +po4a will extract this command from paragraphs (if it is located at +the beginning or the end of a paragraph). +The translators will then have to translate the parameters that are marked +as translatable. + +=item a plus (B<+>) + +As for an asterisk, the command will be extracted if it appear at an +extremity of a block, but the parameters won't be translated separately. +The translator will have to translate the command concatenated to all its +parameters. +This permits to keep more context, and is useful for commands with small +words in parameter, which can have multiple meanings (and translations). + +Note: In this case you don't have to specify which parameters are +translatable, but po4a must know the type and number of parameters. + +=item a minus (B<->) + +In this case, the command won't be extracted from any block. +But if it appears alone on a block, then only the parameters marked as +translatable will be presented to the translator. +This is useful for font commands. These commands should generally not be +separated from their paragraph (to keep the context), but there is no +reason to annoy the translator with them if a whole string is enclosed in +such a command. + +=back + +The I<parameters> argument is a set of [] (to indicate an optional +argument) or {} (to indicate a mandatory argument). +You can place an underscore (_) between these brackets to indicate that +the parameter must be translated. For example: + % po4a: command *chapter [_]{_} + +This indicates that the chapter command has two parameters: an optional +(short title) and a mandatory one, which must both be translated. +If you want to specify that the href command has two mandatory parameters, +that you don't want to translate the URL (first parameter), and that you +don't want this command to be separated from its paragraph (which allow +the translator to move the link in the sentence), you can use: + % po4a: command -href {}{_} + +In this case, the information indicating which arguments must be +translated is only used if a paragraph is only composed of this href +command. + +=item B<% po4a: environment> I<env> I<parameters> + +This permits to define the parameters accepted by the I<env> environment. +This information is latter used to check the number of arguments of the +\begin command, and permit to specify which one must be translated. +The syntax of the I<parameters> argument is the same as described for the +commands. +The first parameter of the \begin command is the name of the environment. +This parameter must not be specified in the list of parameters. Here are +some examples: + % po4a: environment multicols {} + % po4a: environment equation + +As for the commands, I<env> can be preceded by a plus (+) to indicate +that the \begin command must be translated with all its arguments. + +=item B<% po4a: separator> I<env> B<">I<regex>B<"> + +Indicates that an environment should be split according to the given +regular expression. + +The regular expression is delimited by quotes. +It should not create any backreference. +You should use (?:) if you need a group. +It may also need some escapes. + +For example, the LaTeX module uses the "(?:&|\\\\)" regular expression to +translate separately each cell of a table (lines are separated by '\\' and +cells by '&'). + +The notion of environment is expended to the type displayed in the PO file. +This can be used to split on "\\\\" in the first mandatory argument of the +title command. In this case, the environment is title{#1}. + +=item B<% po4a: verbatim environment> I<env> + +Indicate that I<env> is a verbatim environment. +Comments and commands will be ignored in this environment. + +If this environment was not already registered, po4a will consider that +this environment does not take any parameters. + +=back + +=cut + +# Directory name of the main file. +# It is the directory where included files will be searched. +# See read_file. +my $my_dirname; + +# Array of files that should not be included by read_file. +# See read_file. +our @exclude_include; + +my %type_end=('{'=>'}', '['=>']', ' '=>''); + +######################### +#### DEBUGGING STUFF #### +######################### +my %debug=('pretrans' => 0, # see pre-conditioning of translation + 'postrans' => 0, # see post-conditioning of translation + 'translate' => 0, # see translation + 'extract_commands' => 0, # see commands extraction + 'commands' => 0, # see command subroutines + 'environments' => 0, # see environment subroutines + 'translate_buffer' => 0 # see buffer translation + ); + +=head1 WRITING DERIVATE MODULES + +=over 4 + +=item B<pre_trans> + +=cut + +sub pre_trans { + my ($self,$str,$ref,$type)=@_; + # Preformatting, so that translators don't see + # strange chars + my $origstr=$str; + print STDERR "pre_trans($str)=" + if ($debug{'pretrans'}); + + # Accentuated characters + # FIXME: only do this if the encoding is UTF-8? +# $str =~ s/${RE_ESCAPE}`a/à/g; +## $str =~ s/${RE_ESCAPE}c{c}/ç/g; # not in texinfo: @,{c} +# $str =~ s/${RE_ESCAPE}^e/ê/g; +# $str =~ s/${RE_ESCAPE}'e/é/g; +# $str =~ s/${RE_ESCAPE}`e/è/g; +# $str =~ s/${RE_ESCAPE}`u/ù/g; +# $str =~ s/${RE_ESCAPE}"i/ï/g; +# # Non breaking space. FIXME: should we change $\sim$ to ~ +# $str =~ s/~/\xA0/g; # FIXME: not in texinfo: @w{ } + + print STDERR "$str\n" if ($debug{'pretrans'}); + return $str; +} + +=item B<post_trans> + +=cut + +sub post_trans { + my ($self,$str,$ref,$type)=@_; + my $transstr=$str; + + print STDERR "post_trans($str)=" + if ($debug{'postrans'}); + + # Accentuated characters +# $str =~ s/à/${ESCAPE}`a/g; +## $str =~ s/ç/$ESCAPEc{c}/g; # FIXME: not in texinfo +# $str =~ s/ê/${ESCAPE}^e/g; +# $str =~ s/é/${ESCAPE}'e/g; +# $str =~ s/è/${ESCAPE}`e/g; +# $str =~ s/ù/${ESCAPE}`u/g; +# $str =~ s/ï/${ESCAPE}"i/g; +# # Non breaking space. FIXME: should we change ~ to $\sim$ +# $str =~ s/\xA0/~/g; # FIXME: not in texinfo + + print STDERR "$str\n" if ($debug{'postrans'}); + return $str; +} + +# Comments are extracted in the parse function. +# They are stored in the @comments array, and then displayed as a PO +# comment with the first translated string of the paragraph. +my @comments = (); + +=item B<translate> + +Wrapper around Transtractor's translate, with pre- and post-processing +filters. + +Comments of a paragraph are inserted as a PO comment for the first +translated string of this paragraph. + +=cut + +sub translate { + my ($self,$str,$ref,$type) = @_; + my (%options)=@_; + my $origstr=$str; + print STDERR "translate($str)=" + if ($debug{'translate'}); + + return $str unless (defined $str) && length($str); + return $str if ($str eq "\n"); + + $str=pre_trans($self,$str,$ref||$self->{ref},$type); + + # add comments (if any and not already added to the PO) + if (@comments) { + $options{'comment'} .= join('\n', @comments); + + @comments = (); + } + +# FIXME: translate may append a newline, keep the trailing spaces so we can +# recover them. + my $spaces = ""; + if ($options{'wrap'} and $str =~ m/^(.*?)(\s+)$/s) { + $str = $1; + $spaces = $2; + } + + # Translate this + $str = $self->SUPER::translate($str, + $ref||$self->{ref}, + $type || $self->{type}, + %options); + +# FIXME: translate may append a newline, see above + if ($options{'wrap'}) { + chomp $str; + $str .= $spaces; + } + + $str=post_trans($self,$str,$ref||$self->{ref},$type); + + print STDERR "'$str'\n" if ($debug{'translate'}); + return $str; +} + +########################### +### COMMANDS SEPARATION ### +########################### + +=item B<get_leading_command>($buffer) + +This function returns: + +=over 4 + +=item A command name + +If no command is found at the beginning of the given buffer, this string +will be empty. Only commands that can be separated are considered. +The %separated_command hash contains the list of these commands. + +=item A variant + +This indicates if a variant is used. For example, an asterisk (*) can +be added at the end of sections command to specify that they should +not be numbered. In this case, this field will contain "*". If there +is no variant, the field is an empty string. + +=item An array of tuples (type of argument, argument) + +The type of argument can be either '{' (for mandatory arguments) or '[' +(for optional arguments). + +=item The remaining buffer + +The rest of the buffer after the removal of this leading command and +its arguments. If no command is found, the original buffer is not +touched and returned in this field. + +=back + +=cut + +sub get_leading_command { + my ($self, $buffer) = (shift,shift); + my $command = ""; # the command name + my $variant = ""; # a varriant for the command (e.g. an asterisk) + my @args; # array of arguments + print STDERR "get_leading_command($buffer)=" + if ($debug{'extract_commands'}); + + if ($buffer =~ m/^$RE_ESCAPE([[:alnum:]]+)(\*?)(.*)$/s + && defined $separated_command{$1}) { + # The buffer begin by a comand (possibly preceded by some + # whitespaces). + $command = $1; + $variant = $2; + $buffer = $3; + # read the arguments (if any) + while ($buffer =~ m/^\s*$RE_PRE_COMMENT([\[\{])(.*)$/s) { + my $type = $1; + my $arg = ""; + my $count = 1; + $buffer = $2; + # stop reading the buffer when the number of ] (or }) matches the + # the number of [ (or {). + while ($count > 0) { + if ($buffer =~ m/^(.*?$RE_PRE_COMMENT)([\[\]\{\}])(.*)$/s) { + $arg .= $1; + $buffer = $3; + if ($2 eq $type) { + $count++; + } elsif ($2 eq $type_end{$type}) { + $count--; + } + if ($count > 0) { + $arg .= $2 + } + } else { + die wrap_ref_mod($self->{ref}, + "po4a::tex", + dgettext("po4a", "un-balanced %s in '%s'"), + $type, + $buffer); + } + } + push @args, ($type,$arg); + } + } + if (defined $command and length $command) { + # verify the number of arguments + my($check,$reason,$remainder) = check_arg_count($self,$command,\@args); + if (not $check) { + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", + "Error while checking the number of ". + "arguments of the '%s' command: %s")."\n", + $command, $reason); + } + + if (@$remainder) { + # FIXME: we should also keep the spaces to be idempotent + my ($temp,$type,$arg); + while (@$remainder) { + $type = shift @$remainder; + $arg = shift @$remainder; + $temp .= $type.$arg.$type_end{$type}; + # And remove the same number of arguments from @args + pop @args; + pop @args; + } + $buffer = $temp.$buffer; + } + } + + print STDERR "($command,$variant,@args,$buffer)\n" + if ($debug{'extract_commands'}); + return ($command,$variant,\@args,$buffer); +} + +=item B<get_trailing_command>($buffer) + +The same as B<get_leading_command>, but for commands at the end of a buffer. + +=cut + +sub get_trailing_command { + my ($self, $buffer) = (shift,shift); + my $orig_buffer = $buffer; + print STDERR "get_trailing_command($buffer)=" + if ($debug{'extract_commands'}); + + my @args; + my $command = ""; + my $variant = ""; + + # While the buffer ends by }, consider it is a mandatory argument + # and extract this argument. + while ( $buffer =~ m/^(.*$RE_PRE_COMMENT(\{).*)$RE_PRE_COMMENT\}$/s + or $buffer =~ m/^(.*$RE_PRE_COMMENT(\[).*)$RE_PRE_COMMENT\]$/s) { + my $arg = ""; + my $count = 1; + $buffer = $1; + my $type = $2; + # stop reading the buffer when the number of } (or ]) matches the + # the number of { (or [). + while ($count > 0) { + if ($buffer =~ m/^(.*$RE_PRE_COMMENT)([\{\}\[\]])(.*)$/s) { + $arg = $3.$arg; + $buffer = $1; + if ($2 eq $type) { + $count--; + } elsif ($2 eq $type_end{$type}) { + $count++; + } + if ($count > 0) { + $arg = $2.$arg; + } + } else { + die wrap_ref_mod($self->{ref}, + "po4a::tex", + dgettext("po4a", "un-balanced %s in '%s'"), + $type_end{$type}, + $buffer); + } + } + unshift @args, ($type,$arg); + } + + # There should now be a command, maybe followed by an asterisk. + if ($buffer =~ m/^(.*$RE_PRE_COMMENT)$RE_ESCAPE([[:alnum:]]+)(\*?)\s*$/s + && defined $separated_command{$2}) { + $buffer = $1; + $command = $2; + $variant = $3; + my($check,$reason,$remainder) = check_arg_count($self,$command,\@args); + if (not $check) { + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", + "Error while checking the number of ". + "arguments of the '%s' command: %s")."\n", + $command, $reason); + } + if (@$remainder) { + # There are some arguments after the command. + # We can't extract this comand. + $command = ""; + } + } + + # sanitize return values if no command was found. + if (!length($command)) { + $command = ""; + $variant = ""; + undef @args; + $buffer = $orig_buffer; + } +# verify the number of arguments + + print STDERR "($command,$variant,@args,$buffer)\n" + if ($debug{'extract_commands'}); + return ($command,$variant,\@args,$buffer); +} + +=item B<translate_buffer> + +Recursively translate a buffer by separating leading and trailing +commands (those which should be translated separately) from the +buffer. + +If a function is defined in %translate_buffer_env for the current +environment, this function will be used to translate the buffer instead of +translate_buffer(). + +=cut + +our %translate_buffer_env = (); +sub translate_buffer { + my ($self,$buffer,$no_wrap,@env) = (shift,shift,shift,@_); + + if (@env and defined $translate_buffer_env{$env[-1]}) { + return &{$translate_buffer_env{$env[-1]}}($self,$buffer,$no_wrap,@env); + } + + print STDERR "translate_buffer($buffer,$no_wrap,@env)=" + if ($debug{'translate_buffer'}); + + my ($command,$variant) = ("",""); + my $args; + my $translated_buffer = ""; + my $orig_buffer = $buffer; + my $t = ""; # a temporary string + + if ($buffer =~ /^\s*$/s) { + print STDERR "($buffer,@env)\n" + if ($debug{'translate_buffer'}); + return ($buffer, @env); + } + # verbatim blocks. + # Buffers starting by \end{verbatim} are handled after. + if (in_verbatim(@env) and $buffer !~ m/^\n?\Q$ESCAPE\Eend\{$env[-1]\*?\}/) { + if($buffer =~ m/^(.*?)(\n?\Q$ESCAPE\Eend\{$env[-1]\*?\}.*)$/s) { + # end of a verbatim block + my ($begin, $end) = ($1?$1:"", $2); + my ($t1, $t2) = ("", ""); + if (defined $begin) { + $t1 = $self->translate($begin,$self->{ref}, + $env[-1], + "wrap" => 0); + } + ($t2, @env) = translate_buffer($self, $end, $no_wrap, @env); + print STDERR "($t1$t2,@env)\n" + if ($debug{'translate_buffer'}); + return ($t1.$t2, @env); + } else { + $translated_buffer = $self->translate($buffer,$self->{ref}, + $env[-1], + "wrap" => 0); + print STDERR "($translated_buffer,@env)\n" + if ($debug{'translate_buffer'}); + return ($translated_buffer, @env); + } + } + # early detection of verbatim environment + if ($buffer =~ /^($RE_VERBATIM\n?)(.*)$/s and length $2) { + my ($begin, $end) = ($1, $2); + my ($t1, $t2) = ("", ""); + ($t1, @env) = translate_buffer($self, $begin, $no_wrap, @env); + ($t2, @env) = translate_buffer($self, $end, $no_wrap, @env); + + print STDERR "($t1$t2,@env)\n" + if ($debug{'translate_buffer'}); + return ($t1.$t2, @env); + } + # detect \begin and \end (if they are not commented) + if ($buffer =~ /^((?:.*?\n)? # $1 is + (?:[^%] # either not a % + | # or + (?<!\\)(?:\\\\)*\\%)*? # a % preceded by an odd nb of \ + ) # $2 is a \begin{ with the end of the line + (${RE_ESCAPE}(?:begin|end)\{.*)$/sx + and length $1) { + my ($begin, $end) = ($1, $2); + my ($t1, $t2) = ("", ""); + if (is_closed($begin)) { + ($t1, @env) = translate_buffer($self, $begin, $no_wrap, @env); + ($t2, @env) = translate_buffer($self, $end, $no_wrap, @env); + + print STDERR "($t1$t2,@env)\n" + if ($debug{'translate_buffer'}); + return ($t1.$t2, @env); + } + } + + # remove comments from the buffer. + # Comments are stored in an array and shown as comments in the PO. + while ($buffer =~ m/($RE_PRE_COMMENT)$RE_COMMENT([^\n]*)(\n[ \t]*)(.*)$/s) { + my $comment = $2; + my $end = ""; + if ($4 =~ m/^\n/s and $buffer !~ m/^$RE_COMMENT/s) { + # a line with comments, followed by an empty line. + # Keep the empty line, but remove the comment. + # This is an empirical heuristic, but seems to work;) + $end = "\n"; + } + if (defined $comment and $comment !~ /^\s*$/s) { + push @comments, $comment; + } + $buffer =~ s/($RE_PRE_COMMENT)$RE_COMMENT([^\n]*)(\n[ \t]*)/$1$end/s; + } + + # translate leading commands. + do { + # keep the leading space to put them back after the translation of + # the command. + my $spaces = ""; + if ($buffer =~ /^(\s+)(.*?)$/s) { + $spaces = $1; +# $buffer = $2; # FIXME: this also remove trailing spaces!! + $buffer =~ s/^\s*//s; + } + my $buffer_save = $buffer; + ($command, $variant, $args, $buffer) = + get_leading_command($self,$buffer); + if ( (length $command) + and (defined $separated_command{$command}) + and ($separated_command{$command} eq '-') + and ( (not (defined($buffer))) + or ($buffer !~ m/^\s*$/s) )) { + # This command can be separated only if alone on a buffer. + # We need to remove the trailing commands first, and see if it + # will be alone on this buffer. + $buffer = $buffer_save; + $command = ""; + } + if (length($command)) { + # call the command subroutine. + # These command subroutines will probably call translate_buffer + # with the content of each argument that need a translation. + if (defined ($commands{$command})) { + ($t,@env) = &{$commands{$command}}($self,$command,$variant, + $args,\@env,$no_wrap); + $translated_buffer .= $spaces.$t; + # Handle spaces after a command. + $spaces = ""; + if ($buffer =~ /^(\s+)(.*?)$/s) { + $spaces = $1; +# $buffer = $2; # FIXME: this also remove trailing spaces!! + $buffer =~ s/^\s*//s; + } + $translated_buffer .= $spaces; + } else { + die wrap_ref_mod($self->{ref}, + "po4a::tex", + dgettext("po4a", "Unknown command: '%s'"), + $command); + } + } else { + $buffer = $spaces.$buffer; + } + } while (length($command)); + + # array of trailing commands, which will be translated later. + my @trailing_commands = (); + do { + my $spaces = ""; + if ($buffer =~ /^(.*?)(\s+)$/s) { + $buffer = $1; + $spaces = $2; + } + my $buffer_save = $buffer; + ($command, $variant, $args, $buffer) = + get_trailing_command($self,$buffer); + if ( (length $command) + and (defined $separated_command{$command}) + and ($separated_command{$command} eq '-') + and ( (not defined $buffer) + or ($buffer !~ m/^\s*$/s))) { + # We can extract this command. + $command = ""; + $buffer = $buffer_save; + } + if (length($command)) { + unshift @trailing_commands, ($command, $variant, $args, $spaces); + } else { + $buffer .= $spaces; + } + } while (length($command)); + + # Now, $buffer is just a block that can be translated. + + # environment specific treatment + if (@env and defined $env_separators{$env[-1]}) { + my $re_separator = $env_separators{$env[-1]}; + my $buf_begin = ""; +# FIXME: the separator may have to be translated. + while ($buffer =~ m/^(.*?)(\s*$re_separator\s*)(.*)$/s) { + my ($begin, $sep, $end) = ($1, $2, $3); + $buf_begin .= $begin; + if (is_closed($buf_begin)) { + my $t = ""; + ($t, @env) = translate_buffer($self,$buf_begin,$no_wrap,@env); + $translated_buffer .= $t.$sep; + $buf_begin = ""; + } else { + # the command is in a command argument + $buf_begin .= $sep; + } + $buffer = $end; + } + $buffer = $buf_begin . $buffer; + } + + # finally, translate + if (length($buffer)) { + my $wrap = 1; + my ($e1, $e2); + NO_WRAP_LOOP: foreach $e1 (@env) { + foreach $e2 (split(' ', $no_wrap_environments)) { + if ($e1 eq $e2) { + $wrap = 0; + last NO_WRAP_LOOP; + } + } + } + $wrap = 0 if (defined $no_wrap and $no_wrap == 1); + # Keep spaces at the end of the buffer. + my $spaces_end = ""; + if ($buffer =~ /^(.*?)(\s+)$/s) { + $spaces_end = $2; + $buffer = $1; + } + if ($wrap and $buffer =~ s/^(\s+)//s) { + $translated_buffer .= $1; + } + $translated_buffer .= $self->translate($buffer,$self->{ref}, + @env?$env[-1]:"Plain text", + "wrap" => $wrap); + # Restore spaces at the end of the buffer. + $translated_buffer .= $spaces_end; + } + + # append the translation of the trailing commands + while (@trailing_commands) { + my $command = shift @trailing_commands; + my $variant = shift @trailing_commands; + my $args = shift @trailing_commands; + my $spaces = shift @trailing_commands; + if (defined ($commands{$command})) { + ($t,@env) = &{$commands{$command}}($self,$command,$variant, + $args,\@env,$no_wrap); + $translated_buffer .= $t.$spaces; + } else { + die wrap_ref_mod($self->{ref}, + "po4a::tex", + dgettext("po4a", "Unknown command: '%s'"), + $command); + } + } + + print STDERR "($translated_buffer,@env)\n" + if ($debug{'translate_buffer'}); + return ($translated_buffer,@env); +} + +################################ +#### EXTERNAL CUSTOMIZATION #### +################################ + +=item B<read> + +Overload Transtractor's read + +=cut + +sub read { + my $self=shift; + my $filename=shift; + + # keep the directory name of the main file. + $my_dirname = dirname($filename); + + push @{$self->{TT}{doc_in}}, read_file($self, $filename); +} + +=item B<read_file> + +Recursively read a file, appending included files which are not listed in the +@exclude_include array. Included files are searched using the B<kpsewhich> +command from the Kpathsea library. + +Except from the file inclusion part, it is a cut and paste from +Transtractor's read. + +=cut + +# TODO: fix DOS end of lines +sub read_file { + my $self=shift; + my $filename=shift + or croak wrap_mod("po4a::tex", + dgettext("po4a", "Can't read from file without having a filename")); + my $linenum=0; + my @entries=(); + + open (my $in, $filename) + or croak wrap_mod("po4a::tex", + dgettext("po4a", "Can't read from %s: %s"), $filename, $!); + while (defined (my $textline = <$in>)) { + $linenum++; + my $ref="$filename:$linenum"; + # TODO: add support for includeonly + # The next regular expression matches \input or \includes that are + # not commented (but can be preceded by a \%. + while ($textline =~ /^((?:[^%]|(?<!\\)(?:\\\\)*\\%)*) + \\(include|input) + \{([^\{]*)\}(.*)$/x) { + my ($begin,$newfilename,$end) = ($1,$3,$4); + my $tag = $2; + my $include = 1; + foreach my $f (@exclude_include) { + if ($f eq $newfilename) { + $include = 0; + $begin .= "\\$tag"."{$newfilename}"; + $textline = $end; + last; + } + } + if ($include and ($tag eq "include")) { + $begin .= "\\clearpage"; + } + if ($begin !~ /^\s*$/) { + push @entries, ($begin,$ref); + } + if ($include) { + # search the file + open (KPSEA, "kpsewhich " . $newfilename . " |"); + my $newfilepath = <KPSEA>; + + if ($newfilename ne "" and $newfilepath eq "") { + die wrap_mod("po4a::tex", + dgettext("po4a", + "Can't find %s with kpsewhich"), + $filename); + } + + push @entries, read_file($self, + $newfilepath); + if ($tag eq "include") { + $textline = "\\clearpage".$end; + } else { + $textline = $end; + } + } + } + if (length($textline)) { + my @entry=($textline,$ref); + push @entries, @entry; + + # Detect if this file has non-ascii characters + if($self->{TT}{ascii_input}) { + + my $decoder = guess_encoding($textline); + if (!ref($decoder) or $decoder !~ /Encode::XS=/) { + # We have detected a non-ascii line + $self->{TT}{ascii_input} = 0; + # Save the reference for future error message + $self->{TT}{non_ascii_ref} ||= $ref; + } + } + } + } + close $in + or croak wrap_mod("po4a::tex", + dgettext("po4a", "Can't close %s after reading: %s"), $filename, $!); + + return @entries; +} + +=back + + +=over 4 + +=item B<parse_definition_file> + +Subroutine for parsing a file with po4a directives (definitions for +new commands). + +=cut + +sub parse_definition_file { + my ($self,$filename,$only_try)=@_; + my $filename_org = $filename; + + open (KPSEA, "kpsewhich " . $filename . " |"); + $filename = <KPSEA>; + + if (not defined $filename) { + warn wrap_mod("po4a::tex", + dgettext("po4a", "kpsewhich cannot find %s"), $filename_org); + if (defined $only_try && $only_try) { + return; + } else { + exit 1; + } + } + + if (! open (IN,"<$filename")) { + warn wrap_mod("po4a::tex", + dgettext("po4a", "Can't open %s: %s"), $filename, $!); + if (defined $only_try && $only_try) { + return; + } else { + exit 1; + } + } + while (<IN>) { + if (/^\s*%\s*po4a\s*:/) { + parse_definition_line($self, $_); + } + } +} + +=item B<parse_definition_line> + +Parse a definition line of the form "% po4a: ". + +See the B<INLINE CUSTOMIZATION> section for more details. + +=cut + +sub parse_definition_line { + my ($self,$line)=@_; + $line =~ s/^\s*%\s*po4a\s*:\s*//; + + if ($line =~ /^command\s+([-*+]?)(\w+)\s+(.*)$/) { + my $command = $2; + $line = $3; + if ($1) { + $separated_command{$command} = $1; + } + if ($line =~ /^alias\s+(\w+)\s*$/) { + if (defined ($commands{$1})) { + $commands{$command} = $commands{$1}; + $command_parameters{$command} = $command_parameters{$1}; + } else { + die wrap_mod("po4a::tex", + dgettext("po4a", "Cannot use an alias to the unknown command '%s'"), + $2); + } + } elsif ($line =~ /^(-1|\d+),(-1|\d+),(-1|[ 0-9]*),(-1|[ 0-9]*?)\s*$/) { + die wrap_ref_mod($self->{ref}, + "po4a::tex", + dgettext("po4a", "You are using the old ". + "definitions format (%s). ". + "Please update this definition line."), + $_[1]) + } elsif ($line =~ m/^((?:\{_?\}|\[_?\])*)\s*$/) { + register_generic_command("$command,$1"); + } + } elsif ($line =~ /^environment\s+([+]?\w+\*?)(.*)$/) { + my $env = $1; + $line = $2; + if ($line =~ m/^\s*((?:\{_?\}|\[_?\])*)\s*$/) { + register_generic_environment("$env,$1"); + } + } elsif ($line =~ /^separator\s+(\w+(?:\[#[0-9]+\])?)\s+\"(.*)\"\s*$/) { + my $env = $1; # This is not necessarily an environment. + # It can also be smth like 'title[#1]'. + $env_separators{$env} = $2; + } elsif ($line =~ /^verbatim\s+environment\s+(\w+)\s+$/) { + register_verbatim_environment($1); + } +} + +=item B<is_closed> + +=cut + +sub is_closed { + my $paragraph = shift; +# FIXME: [ and ] are more difficult to handle, because it is not easy to detect if it introduce an optional argument + my $tmp = $paragraph; + my $closing = 0; + my $opening = 0; + # FIXME: { and } should not be counted in verbatim blocks + # Remove comments + $tmp =~ s/$RE_PRE_COMMENT$RE_COMMENT.*//mg; + while ($tmp =~ /^.*?$RE_PRE_COMMENT\{(.*)$/s) { + $opening += 1; + $tmp = $1; + } + $tmp = $paragraph; + # Remove comments + $tmp =~ s/$RE_PRE_COMMENT$RE_COMMENT.*//mg; + while ($tmp =~ /^.*?$RE_PRE_COMMENT\}(.*)$/s) { + $closing += 1; + $tmp = $1; + } + return $opening eq $closing; +} + +sub in_verbatim { + foreach my $e1 (@_) { + foreach my $e2 (split(' ', $verbatim_environments)) { + if ($e1 eq $e2) { + return 1; + } + } + } + + return 0; +} + +############################# +#### MAIN PARSE FUNCTION #### +############################# +=item B<parse> + +=cut + +sub parse { + my $self = shift; + my ($line,$ref); + my $paragraph = ""; # Buffer where we put the paragraph while building + my @env = (); # environment stack + my $t = ""; + + LINE: + undef $self->{type}; + ($line,$ref)=$self->shiftline(); + + while (defined($line)) { + chomp($line); + $self->{ref}="$ref"; + + if ($line =~ /^\s*%\s*po4a\s*:/) { + parse_definition_line($self, $line); + goto LINE; + } + + my $closed = is_closed($paragraph); + +#FIXME: what happens if a \begin{verbatim} or \end{verbatim} is in the +# middle of a line. (This is only an issue if the verbatim +# environment contains an un-closed bracket) + if ( ($closed and ($line =~ /^\s*$/ or + $line =~ /^\s*$RE_VERBATIM\s*$/)) + or (in_verbatim(@env) and $line =~ /^\s*\Q$ESCAPE\Eend{$env[-1]}\s*$/) + ) { + # An empty line. This indicates the end of the current + # paragraph. + $paragraph .= $line."\n"; + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + @comments = (); + } + } else { + # continue the same paragraph + $paragraph .= $line."\n"; + } + + # Reinit the loop + ($line,$ref)=$self->shiftline(); + undef $self->{type}; + } + + if (length($paragraph)) { + ($t, @env) = translate_buffer($self,$paragraph,undef,@env); + $self->pushline($t); + $paragraph=""; + } +} # end of parse + +=item B<docheader> + +=back + +=cut + +sub docheader { + return "% This file was generated with po4a. Translate the source file.\n". + "%\n"; +} + + +#################################### +#### DEFINITION OF THE COMMANDS #### +#################################### + +=head1 INTERNAL FUNCTIONS used to write derivated parsers + +Command and environment functions take the following arguments (in +addition to the $self object): + +=over + +=item A command name + +=item A variant + +=item An array of (type, argument) tuples + +=item The current environment + +=back + +The first 3 arguments are extracted by get_leading_command or +get_trailing_command. + +Command and environment functions return the translation of the command +with its arguments and a new environment. + +Environment functions are called when a \begin command is found. They are +called with the \begin command and its arguments. + +The TeX module only proposes one command function and one environment +function: generic_command and generic_environment. + +generic_command uses the information specified by +register_generic_command or by adding definition to the TeX file: + % po4a: command I<command1> I<parameters> + +generic_environment uses the information specified by +register_generic_environment or by adding definition to the TeX file: + % po4a: environment I<env> I<parameters> + +Both functions will only translate the parameters that were specified as +translatable (with a '_'). +generic_environment will append the name of the environment to the +environment stack and generic_command will append the name of the command +followed by an identifier of the parameter (like {#7} or [#2]). + +=cut + +# definition of environment related commands + +$commands{'begin'}= sub { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "begin($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'} || $debug{'environments'}); + my ($t,@e) = ("",()); + + my $envir = $args->[1]; + if (defined($envir) and $envir =~ /^(.*)\*$/) { + $envir = $1; + } + + if (defined($envir) && defined($environments{$envir})) { + ($t, @e) = &{$environments{$envir}}($self,$command,$variant, + $args,$env,$no_wrap); + } else { + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", "unknown environment: '%s'"), + $args->[1]); + } + + print "($t, @e)\n" + if ($debug{'commands'} || $debug{'environments'}); + return ($t, @e); +}; +# Use register_generic to set the type of arguments. The function is then +# overwritten: +register_generic_command("*end,{}"); +$commands{'end'}= sub { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "end($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'} || $debug{'environments'}); + + # verify that this environment was the last pushed environment. + if (!@$env || @$env[-1] ne $args->[1]) { + # a begin may have been hidden in the middle of a translated + # buffer. FIXME: Just warn for now. + warn wrap_ref_mod($self->{'ref'}, "po4a::tex", + dgettext("po4a", "unmatched end of environment '%s'"), + $args->[1]); + } else { + pop @$env; + } + + my ($t,@e) = generic_command($self,$command,$variant,$args,$env,$no_wrap); + + print "($t, @$env)\n" + if ($debug{'commands'} || $debug{'environments'}); + return ($t, @$env); +}; +$separated_command{'begin'} = '*'; + +sub generic_command { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "generic_command($command,$variant,@$args,@$env,$no_wrap)=" + if ($debug{'commands'} || $debug{'environments'}); + + my ($t,@e)=("",()); + my $translated = ""; + + # the number of arguments is checked during the extraction of the + # arguments + + if ( (not (defined $separated_command{$command})) + or $separated_command{$command} ne '+') { + # Use the information from %command_parameters to only translate + # the needed parameters + $translated = "$ESCAPE$command$variant"; + # handle arguments + my @arg_types = @{$command_parameters{$command}{'types'}}; + my @arg_translated = @{$command_parameters{$command}{'translated'}}; + my ($type, $opt); + my @targs = @$args; + my $count = 0; + while (@targs) { + $type = shift @targs; + $opt = shift @targs; + my $have_to_be_translated = 0; +TEST_TYPE: + if ($count >= scalar @arg_types) { + # The number of arguments does not match, + # and a variable number of arguments was not specified + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", + "Wrong number of arguments for ". + "the '%s' command.")."\n", + $command); + } elsif ($type eq $arg_types[$count]) { + $have_to_be_translated = $arg_translated[$count]; + $count ++; + } elsif ($type eq '{' and $arg_types[$count] eq '[') { + # an optionnal argument was not provided, + # try with the next argument. + $count++; + goto TEST_TYPE; + } else { + my $reason = dgettext("po4a", + "An optional argument ". + "was provided, but a mandatory one ". + "is expected."); + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", "Command '%s': %s")."\n", + $command, $reason); + } + if ($have_to_be_translated) { + ($t, @e) = translate_buffer($self,$opt,$no_wrap,(@$env,$command.$type."#".$count.$type_end{$type})); + } else { + $t = $opt; + } + $translated .= $type.$t.$type_end{$type}; + } + } else { + # Translate the command with all its arguments joined + my $tmp = "$ESCAPE$command$variant"; + my ($type, $opt); + while (@$args) { + $type = shift @$args; + $opt = shift @$args; + $tmp .= $type.$opt.$type_end{$type}; + } + @e = @$env; + my $wrap = 1; + $wrap = 0 if (defined $no_wrap and $no_wrap == 1); + $translated = $self->translate($tmp,$self->{ref}, + @e?$e[-1]:"Plain text", + "wrap" => $wrap); + } + + print "($translated, @$env)\n" + if ($debug{'commands'} || $debug{'environments'}); + return ($translated, @$env); +} + +sub register_generic_command { + if ($_[0] =~ m/^(.*),((\{_?\}|\[_?\]| _? )*)$/) { + my $command = $1; + my $arg_types = $2; + if ($command =~ /^([-*+])(.*)$/) { + $command = $2; + $separated_command{$command}=$1; + } + my @types = (); + my @translated = (); + while ( defined $arg_types + and length $arg_types + and $arg_types =~ m/^(?:([\{\[ ])(_?)[\}\] ])(.*)$/) { + push @types, $1; + push @translated, ($2 eq "_")?1:0; + $arg_types = $3; + } + $command_parameters{$command}{'types'} = \@types; + $command_parameters{$command}{'translated'} = \@translated; + $command_parameters{$command}{'nb_args'} = ""; + $commands{$command} = \&generic_command; + } else { + die wrap_mod("po4a::tex", + dgettext("po4a", + "register_generic_command: unsupported ". + "format: '%s'.")."\n", + $_[0]); + } +} + +######################################## +#### DEFINITION OF THE ENVIRONMENTS #### +######################################## +sub generic_environment { + my $self = shift; + my ($command,$variant,$args,$env) = (shift,shift,shift,shift); + my $no_wrap = shift; + print "generic_environment($command,$variant,$args,$env,$no_wrap)=" + if ($debug{'environments'}); + my ($t,@e)=("",()); + my $translated = ""; + + # The first argument (the name of the environment is never translated) + # For the others, @types and @translated are used. + $translated = "$ESCAPE$command$variant"; + my @targs = @$args; + my $type = shift @targs; + my $opt = shift @targs; + my $new_env = $opt; + $translated .= $type.$new_env.$type_end{$type}; + if ( (not (defined $separated_environment{$new_env})) + or $separated_environment{$new_env} ne '+') { + # Use the information from %command_parameters to only translate + # the needed parameters + my @arg_types = @{$environment_parameters{$new_env}{'types'}}; + my @arg_translated = @{$environment_parameters{$new_env}{'translated'}}; + + my $count = 0; + while (@targs) { + $type = shift @targs; + $opt = shift @targs; + my $have_to_be_translated = 0; +TEST_TYPE: + if ($count >= scalar @arg_types) { + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", + "Wrong number of arguments for ". + "the '%s' command.")."\n", + $command); + } elsif ($type eq $arg_types[$count]) { + $have_to_be_translated = $arg_translated[$count]; + $count ++; + } elsif ($type eq '{' and $arg_types[$count] eq '[') { + # an optionnal argument was not provided, + # try with the next argument. + $count++; + goto TEST_TYPE; + } else { + my $reason = dgettext("po4a", + "An optional argument ". + "was provided, but a mandatory one ". + "is expected."); + die wrap_ref_mod($self->{ref}, "po4a::tex", + dgettext("po4a", "Command '%s': %s")."\n", + $command, $reason); + } + + if ($have_to_be_translated) { + ($t, @e) = translate_buffer($self,$opt,$no_wrap,(@$env,$new_env.$type."#".$count.$type_end{$type})); + } else { + $t = $opt; + } + $translated .= $type.$t.$type_end{$type}; + + } + } else { + # Translate the \begin command with all its arguments joined + my ($type, $opt); + my $buf = $translated; + while (@targs) { + $type = shift @targs; + $opt = shift @targs; + $buf .= $type.$opt.$type_end{$type}; + } + @e = @$env; + my $wrap = 1; + $wrap = 0 if $no_wrap == 1; + $translated = $self->translate($buf,$self->{ref}, + @e?$e[-1]:"Plain text", + "wrap" => $wrap); + } + @e = (@$env, $new_env); + + print "($translated,@e)\n" + if ($debug{'environments'}); + return ($translated,@e); +} + + +sub check_arg_count { + my $self = shift; + my $command = shift; + my $args = shift; + my @targs = @$args; + my $check = 1; + my @remainder = (); + my $reason = ""; + my ($type, $arg); + my @arg_types; + + if ($command eq 'begin') { + $type = shift @targs; + # The name of the environment is mandatory + if ( (not defined $type) + or ($type ne '{')) { + $reason = dgettext("po4a", + "The first argument of \\begin is mandatory."); + $check = 0; + } + my $env = shift @targs; + if (not defined $environment_parameters{$env}) { + die wrap_ref_mod($self->{ref},"po4a::tex", + dgettext("po4a", "unknown environment: '%s'"), + $env); + } + @arg_types = @{$environment_parameters{$env}{'types'}}; + } else { + @arg_types = @{$command_parameters{$command}{'types'}}; + } + + my $count = 0; + while ($check and @targs) { + $type = shift @targs; + $arg = shift @targs; +TEST_TYPE: + if ($count >= scalar @arg_types) { + # Too many arguments some will remain + @remainder = ($type, $arg, @targs); + last; + } elsif ($type eq $arg_types[$count]) { + $count ++; + } elsif ($type eq '{' and $arg_types[$count] eq '[') { + # an optionnal argument was not provided, + # try with the next argument. + $count++; + goto TEST_TYPE; + } else { + $check = 0; + $reason = dgettext("po4a", + "An optional argument was ". + "provided, but a mandatory one is expected."); + } + } + + return ($check, $reason, \@remainder); +} + +sub register_generic_environment { + print "register_generic_environment($_[0])\n" + if ($debug{'environments'}); + if ($_[0] =~ m/^(.*),((?:\{_?\}|\[_?\])*)$/) { + my $env = $1; + my $arg_types = $2; + if ($env =~ /^([+])(.*)$/) { + $separated_environment{$2} = $1; + $env = $2; + } + my @types = (); + my @translated = (); + while ( defined $arg_types + and length $arg_types + and $arg_types =~ m/^(?:([\{\[])(_?)[\}\]])(.*)$/) { + push @types, $1; + push @translated, ($2 eq "_")?1:0; + $arg_types = $3; + } + $environment_parameters{$env} = { + 'types' => \@types, + 'translated' => \@translated + }; + $environments{$env} = \&generic_environment; + } +} + +sub register_verbatim_environment { + my $env = shift; + $no_wrap_environments .= " $env"; + $verbatim_environments .= " $env"; + $RE_VERBATIM = "\\\\begin\\{(?:". + join("|", split(/ /, $verbatim_environments)). + ")\\*?\\}"; + register_generic_environment("$env,") + unless (defined $environments{$env}); +} + +#################################### +### INITIALIZATION OF THE PARSER ### +#################################### +sub initialize { + my $self = shift; + my %options = @_; + + $self->{options}{'definitions'}=''; + $self->{options}{'exclude_include'}=''; + $self->{options}{'no_wrap'}=''; + $self->{options}{'verbatim'}=''; + $self->{options}{'debug'}=''; + $self->{options}{'verbose'}=''; + + %debug = (); + # FIXME: %commands and %separated_command should also be restored to their + # default values. + + foreach my $opt (keys %options) { + if ($options{$opt}) { + die wrap_mod("po4a::tex", + dgettext("po4a", "Unknown option: %s"), $opt) + unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options{$opt}; + } + } + + if ($options{'debug'}) { + foreach ($options{'debug'}) { + $debug{$_} = 1; + } + } + + if ($options{'exclude_include'}) { + foreach (split(/:/, $options{'exclude_include'})) { + push @exclude_include, $_; + } + } + + if ($options{'no_wrap'}) { + foreach (split(/,/, $options{'no_wrap'})) { + $no_wrap_environments .= " $_"; + register_generic_environment("$_,") + unless (defined $environments{$_}); + } + } + + if ($options{'verbatim'}) { + foreach (split(/,/, $options{'verbatim'})) { + register_verbatim_environment($_); + } + } + + if ($options{'definitions'}) { + $self->parse_definition_file($options{'definitions'}) + } +} + +=head1 STATUS OF THIS MODULE + +This module needs more tests. + +It was tested on a book and with the Python documentation. + +=head1 TODO LIST + +=over 4 + +=item Automatic detection of new commands + +The TeX module could parse the newcommand arguments and try to guess the +number of arguments, their type and whether or not they should be +translated. + +=item Translation of the environment separator + +When \item is used as an environment separator, the item argument is +attached to the following string. + +=item Some commands should be added to the environment stack + +These commands should be specified by couples. +This could allow to specify commands beginning or ending a verbatim +environment. + +=item Others + +Various other points are tagged TODO in the source. + +=back + +=head1 KNOWN BUGS + +Various points are tagged FIXME in the source. + +=head1 SEE ALSO + +L<Locale::Po4a::LaTeX(3pm)|Locale::Po4a::LaTeX>, +L<Locale::Po4a::TransTractor(3pm)|Locale::Po4a::TransTractor>, +L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Nicolas François <nicolas.francois@centraliens.net> + +=head1 COPYRIGHT AND LICENSE + +Copyright 2004, 2005 by Nicolas FRANÇOIS <nicolas.francois@centraliens.net>. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +1; Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Xml.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Xml.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Xml.pm (revision 2840) @@ -0,0 +1,2082 @@ +#!/usr/bin/perl + +# Po4a::Xml.pm +# +# extract and translate translatable strings from XML documents. +# +# This code extracts plain text from tags and attributes from generic +# XML documents, and it can be used as a base to build modules for +# XML-based documents. +# +# Copyright (c) 2004 by Jordi Vilalta <jvprat@gmail.com> +# Copyright (c) 2008-2009 by Nicolas François <nicolas.francois@centraliens.net> +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Xml - convert XML documents and derivates from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Xml is a module to help the translation of XML documents into +other [human] languages. It can also be used as a base to build modules for +XML-based documents. + +=cut + +package Locale::Po4a::Xml; + +use 5.006; +use strict; +use warnings; + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(new initialize @tag_types); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; +use Carp qw(croak); +use File::Basename; +use File::Spec; + +#It will mantain the path from the root tag to the current one +my @path; + +#It will contain a list of external entities and their attached paths +my %entities; + +my @comments; +my %translate_options_cache; + +my $_shiftline_in_comment = 0; +sub shiftline { + my $self = shift; + # call Transtractor's shiftline + my ($line,$ref) = $self->SUPER::shiftline(); + return ($line,$ref) if (not defined $line); + + if ($self->{options}{'includeexternal'}) { + my $tmp; + + for my $k (keys %entities) { + if ($line =~ m/^(.*?)&$k;(.*)$/s) { + my ($before, $after) = ($1, $2); + my $linenum=0; + my @textentries; + + $tmp = $before; + my $tmp_in_comment = 0; + if ($_shiftline_in_comment) { + if ($before =~ m/^.*?-->(.*)$/s) { + $tmp = $1; + $tmp_in_comment = 0; + } else { + $tmp_in_comment = 1; + } + } + if ($tmp_in_comment == 0) { + while ($tmp =~ m/^.*?<!--.*?-->(.*)$/s) { + $tmp = $1; + } + if ($tmp =~ m/<!--/s) { + $tmp_in_comment = 1; + } + } + next if ($tmp_in_comment); + + open (my $in, $entities{$k}) + or croak wrap_mod("po4a::xml", + dgettext("po4a", "Can't read from %s: %s"), + $entities{$k}, $!); + while (defined (my $textline = <$in>)) { + $linenum++; + my $textref=$entities{$k}.":$linenum"; + push @textentries, ($textline,$textref); + } + close $in + or croak wrap_mod("po4a::xml", + dgettext("po4a", "Can't close %s after reading: %s"), + $entities{$k}, $!); + + push @textentries, ($after, $ref); + $line = $before.(shift @textentries); + $ref .= " ".(shift @textentries); + $self->unshiftline(@textentries); + } + } + + $tmp = $line; + if ($_shiftline_in_comment) { + if ($line =~ m/^.*?-->(.*)$/s) { + $tmp = $1; + $_shiftline_in_comment = 0; + } else { + $_shiftline_in_comment = 1; + } + } + if ($_shiftline_in_comment == 0) { + while ($tmp =~ m/^.*?<!--.*?-->(.*)$/s) { + $tmp = $1; + } + if ($tmp =~ m/<!--/s) { + $_shiftline_in_comment = 1; + } + } + } + + return ($line,$ref); +} + +sub read { + my ($self,$filename)=@_; + push @{$self->{DOCPOD}{infile}}, $filename; + $self->Locale::Po4a::TransTractor::read($filename); +} + +sub parse { + my $self=shift; + map {$self->parse_file($_)} @{$self->{DOCPOD}{infile}}; +} + +# @save_holders is a stack of references to ('paragraph', 'translation', +# 'sub_translations', 'open', 'close', 'folded_attributes') hashes, where: +# paragraph is a reference to an array (see paragraph in the +# treat_content() subroutine) of strings followed by +# references. It contains the @paragraph array as it was +# before the processing was interrupted by a tag instroducing +# a placeholder. +# translation is the translation of this level up to now +# sub_translations is a reference to an array of strings containing the +# translations which must replace the placeholders. +# open is the tag which opened the placeholder. +# close is the tag which closed the placeholder. +# folded_attributes is an hash of tags with their attributes (<tag attrs=...> +# strings), referenced by the folded tag id, which should +# replace the <tag po4a-id=id> strings in the current +# translation. +# +# If @save_holders only has 1 holder, then we are not processing the +# content of an holder, we are translating the document. +my @save_holders; + + +# If we are at the bottom of the stack and there is no <placeholder ...> in +# the current translation, we can push the translation in the translated +# document. +# Otherwise, we keep the translation in the current holder. +sub pushline { + my ($self, $line) = (shift, shift); + + my $holder = $save_holders[$#save_holders]; + my $translation = $holder->{'translation'}; + $translation .= $line; + + while ( %{$holder->{folded_attributes}} + and $translation =~ m/^(.*)<([^>]+?)\s+po4a-id=([0-9]+)>(.*)$/s) { + my $begin = $1; + my $tag = $2; + my $id = $3; + my $end = $4; + if (defined $holder->{folded_attributes}->{$id}) { + # TODO: check if the tag is the same + $translation = $begin.$holder->{folded_attributes}->{$id}.$end; + delete $holder->{folded_attributes}->{$id}; + } else { + # TODO: It will be hard to identify the location. + # => find a way to retrieve the reference. + die wrap_mod("po4a::xml", dgettext("po4a", "'po4a-id=%d' in the translation does not exist in the original string (or 'po4a-id=%d' used twice in the translation)."), $id, $id); + } + } +# TODO: check that %folded_attributes is empty at some time +# => in translate_paragraph? + + if ( ($#save_holders > 0) + or ($translation =~ m/<placeholder\s+type="[^"]+"\s+id="(\d+)"\s*\/>/s)) { + $holder->{'translation'} = $translation; + } else { + $self->SUPER::pushline($translation); + $holder->{'translation'} = ''; + } +} + +=head1 TRANSLATING WITH PO4A::XML + +This module can be used directly to handle generic XML documents. This will +extract all tag's content, and no attributes, since it's where the text is +written in most XML based documents. + +There are some options (described in the next section) that can customize +this behavior. If this doesn't fit to your document format you're encouraged +to write your own module derived from this, to describe your format's details. +See the section B<WRITING DERIVATE MODULES> below, for the process description. + +=cut + +# +# Parse file and translate it +# +sub parse_file { + my ($self,$filename) = @_; + my $eof = 0; + + while (!$eof) { + # We get all the text until the next breaking tag (not + # inline) and translate it + $eof = $self->treat_content; + if (!$eof) { + # And then we treat the following breaking tag + $eof = $self->treat_tag; + } + } +} + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +The global debug option causes this module to show the excluded strings, in +order to see if it skips something important. + +These are this module's particular options: + +=over 4 + +=item B<nostrip> + +Prevents it to strip the spaces around the extracted strings. + +=item B<wrap> + +Canonizes the string to translate, considering that whitespaces are not +important, and wraps the translated document. This option can be overridden +by custom tag options. See the "tags" option below. + +=item B<caseinsensitive> + +It makes the tags and attributes searching to work in a case insensitive +way. If it's defined, it will treat E<lt>BooKE<gt>laNG and E<lt>BOOKE<gt>Lang as E<lt>bookE<gt>lang. + +=item B<includeexternal> + +When defined, external entities are included in the generated (translated) +document, and for the extraction of strings. If it's not defined, you +will have to translate external entities separately as independent +documents. + +=item B<ontagerror> + +This option defines the behavior of the module when it encounter a invalid +XML syntax (a closing tag which does not match the last opening tag, or a +tag's attribute without value). +It can take the following values: + +=over + +=item I<fail> + +This is the default value. +The module will exit with an error. + +=item I<warn> + +The module will continue, and will issue a warning. + +=item I<silent> + +The module will continue without any warnings. + +=back + +Be careful when using this option. +It is generally recommended to fix the input file. + +=item B<tagsonly> + +Extracts only the specified tags in the "tags" option. Otherwise, it +will extract all the tags except the ones specified. + +Note: This option is deprecated. + +=item B<doctype> + +String that will try to match with the first line of the document's doctype +(if defined). If it doesn't, a warning will indicate that the document +might be of a bad type. + +=item B<addlang> + +String indicating the path (e.g. E<lt>bbbE<gt>E<lt>aaaE<gt>) of a tag +where a lang="..." attribute shall be added. The language will be defined +as the basename of the PO file without any .po extension. + +=item B<tags> + +Space-separated list of tags you want to translate or skip. By default, +the specified tags will be excluded, but if you use the "tagsonly" option, +the specified tags will be the only ones included. The tags must be in the +form E<lt>aaaE<gt>, but you can join some (E<lt>bbbE<gt>E<lt>aaaE<gt>) to say that the content of +the tag E<lt>aaaE<gt> will only be translated when it's into a E<lt>bbbE<gt> tag. + +You can also specify some tag options by putting some characters in front of +the tag hierarchy. For example, you can put 'w' (wrap) or 'W' (don't wrap) +to override the default behavior specified by the global "wrap" option. + +Example: WE<lt>chapterE<gt>E<lt>titleE<gt> + +Note: This option is deprecated. +You should use the B<translated> and B<untranslated> options instead. + +=item B<attributes> + +Space-separated list of tag's attributes you want to translate. You can +specify the attributes by their name (for example, "lang"), but you can +prefix it with a tag hierarchy, to specify that this attribute will only be +translated when it's into the specified tag. For example: E<lt>bbbE<gt>E<lt>aaaE<gt>lang +specifies that the lang attribute will only be translated if it's into an +E<lt>aaaE<gt> tag, and it's into a E<lt>bbbE<gt> tag. + +=item B<foldattributes> + +Do not translate attributes in inline tags. +Instead, replace all attributes of a tag by po4a-id=<id>. + +This is useful when attributes shall not be translated, as this simplifies the +strings for translators, and avoids typos. + +=item B<customtag> + +Space-separated list of tags which should not be treated as tags. +These tags are treated as inline, and do not need to be closed. + +=item B<break> + +Space-separated list of tags which should break the sequence. +By default, all tags break the sequence. + +The tags must be in the form <aaa>, but you can join some +(<bbb><aaa>), if a tag (<aaa>) should only be considered +when it's into another tag (<bbb>). + +=item B<inline> + +Space-separated list of tags which should be treated as inline. +By default, all tags break the sequence. + +The tags must be in the form <aaa>, but you can join some +(<bbb><aaa>), if a tag (<aaa>) should only be considered +when it's into another tag (<bbb>). + +=item B<placeholder> + +Space-separated list of tags which should be treated as placeholders. +Placeholders do not break the sequence, but the content of placeholders is +translated separately. + +The location of the placeholder in its block will be marked with a string +similar to: + + <placeholder type=\"footnote\" id=\"0\"/> + +The tags must be in the form <aaa>, but you can join some +(<bbb><aaa>), if a tag (<aaa>) should only be considered +when it's into another tag (<bbb>). + +=item B<nodefault> + +Space separated list of tags that the module should not try to set by +default in any category. + +=item B<cpp> + +Support C preprocessor directives. +When this option is set, po4a will consider preprocessor directives as +paragraph separators. +This is important if the XML file must be preprocessed because otherwise +the directives may be inserted in the middle of lines if po4a consider it +belong to the current paragraph, and they won't be recognized by the +preprocessor. +Note: the preprocessor directives must only appear between tags +(they must not break a tag). + +=item B<translated> + +Space-separated list of tags you want to translate. + +The tags must be in the form <aaa>, but you can join some +(<bbb><aaa>), if a tag (<aaa>) should only be considered +when it's into another tag (<bbb>). + +You can also specify some tag options by putting some characters in front of +the tag hierarchy. For example, you can put 'w' (wrap) or 'W' (don't wrap) +to override the default behavior specified by the global "wrap" option. + +Example: WE<lt>chapterE<gt>E<lt>titleE<gt> + +=item B<untranslated> + +Space-separated list of tags you do not want to translate. + +The tags must be in the form <aaa>, but you can join some +(<bbb><aaa>), if a tag (<aaa>) should only be considered +when it's into another tag (<bbb>). + +=item B<defaulttranslateoption> + +The default categories for tags that are not in any of the translated, +untranslated, break, inline, or placeholder. + +This is a set of letters: + +=over + +=item I<w> + +Tags should be translated and content can be re-wrapped. + +=item I<W> + +Tags should be translated and content should not be re-wrapped. + +=item I<i> + +Tags should be translated inline. + +=item I<p> + +Tags should be translated as placeholders. + +=back + +=back + +=cut +# TODO: defaulttranslateoption +# w => indicate that it is only valid for translatable tags and do not +# care about inline/break/placeholder? +# ... + +sub initialize { + my $self = shift; + my %options = @_; + + # Reset the path + @path = (); + + # Initialize the stack of holders + my @paragraph = (); + my @sub_translations = (); + my %folded_attributes; + my %holder = ('paragraph' => \@paragraph, + 'translation' => "", + 'sub_translations' => \@sub_translations, + 'folded_attributes' => \%folded_attributes); + @save_holders = (\%holder); + + $self->{options}{'addlang'}=0; + $self->{options}{'nostrip'}=0; + $self->{options}{'wrap'}=0; + $self->{options}{'caseinsensitive'}=0; + $self->{options}{'tagsonly'}=0; + $self->{options}{'tags'}=''; + $self->{options}{'break'}=''; + $self->{options}{'translated'}=''; + $self->{options}{'untranslated'}=''; + $self->{options}{'defaulttranslateoption'}=''; + $self->{options}{'attributes'}=''; + $self->{options}{'foldattributes'}=0; + $self->{options}{'inline'}=''; + $self->{options}{'placeholder'}=''; + $self->{options}{'customtag'}=''; + $self->{options}{'doctype'}=''; + $self->{options}{'nodefault'}=''; + $self->{options}{'includeexternal'}=0; + $self->{options}{'ontagerror'}="fail"; + $self->{options}{'cpp'}=0; + + $self->{options}{'verbose'}=''; + $self->{options}{'debug'}=''; + + foreach my $opt (keys %options) { + if ($options{$opt}) { + die wrap_mod("po4a::xml", + dgettext("po4a", "Unknown option: %s"), $opt) + unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options{$opt}; + } + } + # Default options set by modules. Forbidden for users. + $self->{options}{'_default_translated'}=''; + $self->{options}{'_default_untranslated'}=''; + $self->{options}{'_default_break'}=''; + $self->{options}{'_default_inline'}=''; + $self->{options}{'_default_placeholder'}=''; + $self->{options}{'_default_attributes'}=''; + $self->{options}{'_default_customtag'}=''; + + #It will maintain the list of the translatable tags + $self->{tags}=(); + $self->{translated}=(); + $self->{untranslated}=(); + #It will maintain the list of the translatable attributes + $self->{attributes}=(); + #It will maintain the list of the breaking tags + $self->{break}=(); + #It will maintain the list of the inline tags + $self->{inline}=(); + #It will maintain the list of the placeholder tags + $self->{placeholder}=(); + #It will maintain the list of the customtag tags + $self->{customtag}=(); + #list of the tags that must not be set in the tags or inline category + #by this module or sub-module (unless specified in an option) + $self->{nodefault}=(); + + $self->treat_options; + + # Clear cache + %translate_options_cache=(); +} + +=head1 WRITING DERIVATE MODULES + +=head2 DEFINE WHAT TAGS AND ATTRIBUTES TO TRANSLATE + +The simplest customization is to define which tags and attributes you want +the parser to translate. This should be done in the initialize function. +First you should call the main initialize, to get the command-line options, +and then, append your custom definitions to the options hash. If you want +to treat some new options from command line, you should define them before +calling the main initialize: + + $self->{options}{'new_option'}=''; + $self->SUPER::initialize(%options); + $self->{options}{'_default_translated'}.=' <p> <head><title>'; + $self->{options}{'attributes'}.=' <p>lang id'; + $self->{options}{'_default_inline'}.=' <br>'; + $self->treat_options; + +You should use the B<_default_inline>, B<_default_break>, +B<_default_placeholder>, B<_default_translated>, B<_default_untranslated>, +and B<_default_attributes> options in derivated modules. This allow users +to override the default behavior defined in your module with command line +options. + +=head2 OVERRIDING THE found_string FUNCTION + +Another simple step is to override the function "found_string", which +receives the extracted strings from the parser, in order to translate them. +There you can control which strings you want to translate, and perform +transformations to them before or after the translation itself. + +It receives the extracted text, the reference on where it was, and a hash +that contains extra information to control what strings to translate, how +to translate them and to generate the comment. + +The content of these options depends on the kind of string it is (specified in an +entry of this hash): + +=over + +=item type="tag" + +The found string is the content of a translatable tag. The entry "tag_options" +contains the option characters in front of the tag hierarchy in the module +"tags" option. + +=item type="attribute" + +Means that the found string is the value of a translatable attribute. The +entry "attribute" has the name of the attribute. + +=back + +It must return the text that will replace the original in the translated +document. Here's a basic example of this function: + + sub found_string { + my ($self,$text,$ref,$options)=@_; + $text = $self->translate($text,$ref,"type ".$options->{'type'}, + 'wrap'=>$self->{options}{'wrap'}); + return $text; + } + +There's another simple example in the new Dia module, which only filters +some strings. + +=cut + +sub found_string { + my ($self,$text,$ref,$options)=@_; + + if ($text =~ m/^\s*$/s) { + return $text; + } + + my $comment; + my $wrap = $self->{options}{'wrap'}; + + if ($options->{'type'} eq "tag") { + $comment = "Content of: ".$self->get_path; + + if($options->{'tag_options'} =~ /w/) { + $wrap = 1; + } + if($options->{'tag_options'} =~ /W/) { + $wrap = 0; + } + } elsif ($options->{'type'} eq "attribute") { + $comment = "Attribute '".$options->{'attribute'}."' of: ".$self->get_path; + } elsif ($options->{'type'} eq "CDATA") { + $comment = "CDATA"; + $wrap = 0; + } else { + die wrap_ref_mod($ref, "po4a::xml", dgettext("po4a", "Internal error: unknown type identifier '%s'."), $options->{'type'}); + } + $text = $self->translate($text,$ref,$comment,'wrap'=>$wrap, comment => $options->{'comments'}); + return $text; +} + +=head2 MODIFYING TAG TYPES (TODO) + +This is a more complex one, but it enables a (almost) total customization. +It's based in a list of hashes, each one defining a tag type's behavior. The +list should be sorted so that the most general tags are after the most +concrete ones (sorted first by the beginning and then by the end keys). To +define a tag type you'll have to make a hash with the following keys: + +=over 4 + +=item B<beginning> + +Specifies the beginning of the tag, after the "E<lt>". + +=item B<end> + +Specifies the end of the tag, before the "E<gt>". + +=item B<breaking> + +It says if this is a breaking tag class. A non-breaking (inline) tag is one +that can be taken as part of the content of another tag. It can take the +values false (0), true (1) or undefined. If you leave this undefined, you'll +have to define the f_breaking function that will say whether a concrete tag of +this class is a breaking tag or not. + +=item B<f_breaking> + +It's a function that will tell if the next tag is a breaking one or not. It +should be defined if the B<breaking> option is not. + +=item B<f_extract> + +If you leave this key undefined, the generic extraction function will have to +extract the tag itself. It's useful for tags that can have other tags or +special structures in them, so that the main parser doesn't get mad. This +function receives a boolean that says if the tag should be removed from the +input stream or not. + +=item B<f_translate> + +This function receives the tag (in the get_string_until() format) and returns +the translated tag (translated attributes or all needed transformations) as a +single string. + +=back + +=cut + +##### Generic XML tag types #####' + +our @tag_types = ( + { beginning => "!--#", + end => "--", + breaking => 0, + f_extract => \&tag_extract_comment, + f_translate => \&tag_trans_comment}, + { beginning => "!--", + end => "--", + breaking => 0, + f_extract => \&tag_extract_comment, + f_translate => \&tag_trans_comment}, + { beginning => "?xml", + end => "?", + breaking => 1, + f_translate => \&tag_trans_xmlhead}, + { beginning => "?", + end => "?", + breaking => 1, + f_translate => \&tag_trans_procins}, + { beginning => "!DOCTYPE", + end => "", + breaking => 1, + f_extract => \&tag_extract_doctype, + f_translate => \&tag_trans_doctype}, + { beginning => "![CDATA[", + end => "]]", + breaking => 1, + f_extract => \&CDATA_extract, + f_translate => \&CDATA_trans}, + { beginning => "/", + end => "", + f_breaking => \&tag_break_close, + f_translate => \&tag_trans_close}, + { beginning => "", + end => "/", + f_breaking => \&tag_break_alone, + f_translate => \&tag_trans_alone}, + { beginning => "", + end => "", + f_breaking => \&tag_break_open, + f_translate => \&tag_trans_open} +); + +sub tag_extract_comment { + my ($self,$remove)=(shift,shift); + my ($eof,@tag)=$self->get_string_until('-->',{include=>1,remove=>$remove}); + return ($eof,@tag); +} + +sub tag_trans_comment { + my ($self,@tag)=@_; + return $self->join_lines(@tag); +} + +sub tag_trans_xmlhead { + my ($self,@tag)=@_; + + # We don't have to translate anything from here: throw away references + my $tag = $self->join_lines(@tag); + $tag =~ /encoding=(("|')|)(.*?)(\s|\2)/s; + my $in_charset=$3; + $self->detected_charset($in_charset); + my $out_charset=$self->get_out_charset; + + if (defined $in_charset) { + $tag =~ s/$in_charset/$out_charset/; + } else { + if ($tag =~ m/standalone/) { + $tag =~ s/(standalone)/encoding="$out_charset" $1/; + } else { + $tag.= " encoding=\"$out_charset\""; + } + } + + return $tag; +} + +sub tag_trans_procins { + my ($self,@tag)=@_; + return $self->join_lines(@tag); +} + +sub tag_extract_doctype { + my ($self,$remove)=(shift,shift); + + # Check if there is an internal subset (between []). + my ($eof,@tag)=$self->get_string_until('>',{include=>1,unquoted=>1}); + my $parity = 0; + my $paragraph = ""; + map { $parity = 1 - $parity; $paragraph.= $parity?$_:""; } @tag; + my $found = 0; + if ($paragraph =~ m/<.*\[.*</s) { + $found = 1 + } + + if (not $found) { + ($eof,@tag)=$self->get_string_until('>',{include=>1,remove=>$remove,unquoted=>1}); + } else { + ($eof,@tag)=$self->get_string_until(']\s*>',{include=>1,remove=>$remove,unquoted=>1,regex=>1}); + } + return ($eof,@tag); +} + +sub tag_trans_doctype { +# This check is not really reliable. There are system and public +# identifiers. Only the public one could be checked reliably. + my ($self,@tag)=@_; + if (defined $self->{options}{'doctype'} ) { + my $doctype = $self->{options}{'doctype'}; + if ( $tag[0] !~ /\Q$doctype\E/i ) { + warn wrap_ref_mod($tag[1], "po4a::xml", dgettext("po4a", "Bad document type. '%s' expected. You can fix this warning with a -o doctype option, or ignore this check with -o doctype=\"\"."), $doctype); + } + } + my $i = 0; + my $basedir = $tag[1]; + $basedir =~ s/:[0-9]+$//; + $basedir = dirname($basedir); + + while ( $i < $#tag ) { + my $t = $tag[$i]; + my $ref = $tag[$i+1]; + if ( $t =~ /^(\s*<!ENTITY\s+)(.*)$/is ) { + my $part1 = $1; + my $part2 = $2; + my $includenow = 0; + my $file = 0; + my $name = ""; + if ($part2 =~ /^(%\s+)(.*)$/s ) { + $part1.= $1; + $part2 = $2; + $includenow = 1; + } + $part2 =~ /^(\S+)(\s+)(.*)$/s; + $name = $1; + $part1.= $1.$2; + $part2 = $3; + if ( $part2 =~ /^(SYSTEM\s+)(.*)$/is ) { + $part1.= $1; + $part2 = $2; + $file = 1; + if ($self->{options}{'includeexternal'}) { + $entities{$name} = $part2; + $entities{$name} =~ s/^"?(.*?)".*$/$1/s; + $entities{$name} = File::Spec->catfile($basedir, $entities{$name}); + } + } + if ((not $file) and (not $includenow)) { + if ($part2 =~ m/^\s*(["'])(.*)\1(\s*>.*)$/s) { + my $comment = "Content of the $name entity"; + my $quote = $1; + my $text = $2; + $part2 = $3; + $text = $self->translate($text, + $ref, + $comment, + 'wrap'=>1); + $t = $part1."$quote$text$quote$part2"; + } + } +# print $part1."\n"; +# print $name."\n"; +# print $part2."\n"; + } + $tag[$i] = $t; + $i += 2; + } + return $self->join_lines(@tag); +} + +sub tag_break_close { + my ($self,@tag)=@_; + my $struct = $self->get_path; + my $options = $self->get_translate_options($struct); + if ($options =~ m/[ip]/) { + return 0; + } else { + return 1; + } +} + +sub tag_trans_close { + my ($self,@tag)=@_; + my $name = $self->get_tag_name(@tag); + + my $test = pop @path; + if (!defined($test) || $test ne $name ) { + my $ontagerror = $self->{options}{'ontagerror'}; + if ($ontagerror eq "warn") { + warn wrap_ref_mod($tag[1], "po4a::xml", dgettext("po4a", "Unexpected closing tag </%s> found. The main document may be wrong. Continuing..."), $name); + } elsif ($ontagerror ne "silent") { + die wrap_ref_mod($tag[1], "po4a::xml", dgettext("po4a", "Unexpected closing tag </%s> found. The main document may be wrong."), $name); + } + } + return $self->join_lines(@tag); +} + +sub CDATA_extract { + my ($self,$remove)=(shift,shift); + my ($eof, @tag) = $self->get_string_until(']]>',{include=>1,unquoted=>0,remove=>$remove}); + + return ($eof, @tag); +} + +sub CDATA_trans { + my ($self,@tag)=@_; + return $self->found_string($self->join_lines(@tag), + $tag[1], + {'type' => "CDATA"}); +} + +sub tag_break_alone { + my ($self,@tag)=@_; + my $struct = $self->get_path($self->get_tag_name(@tag)); + if ($self->get_translate_options($struct) =~ m/i/) { + return 0; + } else { + return 1; + } +} + +sub tag_trans_alone { + my ($self,@tag)=@_; + my $name = $self->get_tag_name(@tag); + push @path, $name; + + $name = $self->treat_attributes(@tag); + + pop @path; + return $name; +} + +sub tag_break_open { + my ($self,@tag)=@_; + my $struct = $self->get_path($self->get_tag_name(@tag)); + my $options = $self->get_translate_options($struct); + if ($options =~ m/[ip]/) { + return 0; + } else { + return 1; + } +} + +sub tag_trans_open { + my ($self,@tag)=@_; + my $name = $self->get_tag_name(@tag); + push @path, $name; + + $name = $self->treat_attributes(@tag); + + if (defined $self->{options}{'addlang'}) { + my $struct = $self->get_path(); + if ($struct eq $self->{options}{'addlang'}) { + $name .= ' lang="'.$self->{TT}{po_in}->{lang}.'"'; + } + } + + return $name; +} + +##### END of Generic XML tag types ##### + +=head1 INTERNAL FUNCTIONS used to write derivated parsers + +=head2 WORKING WITH TAGS + +=over 4 + +=item get_path() + +This function returns the path to the current tag from the document's root, +in the form E<lt>htmlE<gt>E<lt>bodyE<gt>E<lt>pE<gt>. + +An additional array of tags (without brackets) can be passed as argument. +These path elements are added to the end of the current path. + +=cut + +sub get_path { + my $self = shift; + my @add = @_; + if ( @path > 0 or @add > 0 ) { + return "<".join("><",@path,@add).">"; + } else { + return "outside any tag (error?)"; + } +} + +=item tag_type() + +This function returns the index from the tag_types list that fits to the next +tag in the input stream, or -1 if it's at the end of the input file. + +=cut + +sub tag_type { + my $self = shift; + my ($line,$ref) = $self->shiftline(); + my ($match1,$match2); + my $found = 0; + my $i = 0; + + if (!defined($line)) { return -1; } + + $self->unshiftline($line,$ref); + my ($eof,@lines) = $self->get_string_until(">",{include=>1,unquoted=>1}); + my $line2 = $self->join_lines(@lines); + while (!$found && $i < @tag_types) { + ($match1,$match2) = ($tag_types[$i]->{beginning},$tag_types[$i]->{end}); + if ($line =~ /^<\Q$match1\E/) { + if (!defined($tag_types[$i]->{f_extract})) { +#print substr($line2,length($line2)-1-length($match2),1+length($match2))."\n"; + if (defined($line2) and $line2 =~ /\Q$match2\E>$/) { + $found = 1; +#print "YES: <".$match1." ".$match2.">\n"; + } else { +#print "NO: <".$match1." ".$match2.">\n"; + $i++; + } + } else { + $found = 1; + } + } else { + $i++; + } + } + if (!$found) { + #It should never enter here, unless you undefine the most + #general tags (as <...>) + die "po4a::xml: Unknown tag type: ".$line."\n"; + } else { + return $i; + } +} + +=item extract_tag($$) + +This function returns the next tag from the input stream without the beginning +and end, in an array form, to maintain the references from the input file. It +has two parameters: the type of the tag (as returned by tag_type) and a +boolean, that indicates if it should be removed from the input stream. + +=cut + +sub extract_tag { + my ($self,$type,$remove) = (shift,shift,shift); + my ($match1,$match2) = ($tag_types[$type]->{beginning},$tag_types[$type]->{end}); + my ($eof,@tag); + if (defined($tag_types[$type]->{f_extract})) { + ($eof,@tag) = &{$tag_types[$type]->{f_extract}}($self,$remove); + } else { + ($eof,@tag) = $self->get_string_until($match2.">",{include=>1,remove=>$remove,unquoted=>1}); + } + $tag[0] =~ /^<\Q$match1\E(.*)$/s; + $tag[0] = $1; + $tag[$#tag-1] =~ /^(.*)\Q$match2\E>$/s; + $tag[$#tag-1] = $1; + return ($eof,@tag); +} + +=item get_tag_name(@) + +This function returns the name of the tag passed as an argument, in the array +form returned by extract_tag. + +=cut + +sub get_tag_name { + my ($self,@tag)=@_; + $tag[0] =~ /^(\S*)/; + return $1; +} + +=item breaking_tag() + +This function returns a boolean that says if the next tag in the input stream +is a breaking tag or not (inline tag). It leaves the input stream intact. + +=cut + +sub breaking_tag { + my $self = shift; + my $break; + + my $type = $self->tag_type; + if ($type == -1) { return 0; } + +#print "TAG TYPE = ".$type."\n"; + $break = $tag_types[$type]->{breaking}; + if (!defined($break)) { + # This tag's breaking depends on its content + my ($eof,@lines) = $self->extract_tag($type,0); + $break = &{$tag_types[$type]->{f_breaking}}($self,@lines); + } +#print "break = ".$break."\n"; + return $break; +} + +=item treat_tag() + +This function translates the next tag from the input stream. Using each +tag type's custom translation functions. + +=cut + +sub treat_tag { + my $self = shift; + my $type = $self->tag_type; + + my ($match1,$match2) = ($tag_types[$type]->{beginning},$tag_types[$type]->{end}); + my ($eof,@lines) = $self->extract_tag($type,1); + + $lines[0] =~ /^(\s*)(.*)$/s; + my $space1 = $1; + $lines[0] = $2; + $lines[$#lines-1] =~ /^(.*?)(\s*)$/s; + my $space2 = $2; + $lines[$#lines-1] = $1; + + # Calling this tag type's specific handling (translation of + # attributes...) + my $line = &{$tag_types[$type]->{f_translate}}($self,@lines); + $self->pushline("<".$match1.$space1.$line.$space2.$match2.">"); + return $eof; +} + +=item tag_in_list($@) + +This function returns a string value that says if the first argument (a tag +hierarchy) matches any of the tags from the second argument (a list of tags +or tag hierarchies). If it doesn't match, it returns 0. Else, it returns the +matched tag's options (the characters in front of the tag) or 1 (if that tag +doesn't have options). + +=back + +=cut +sub tag_in_list ($$$) { + my ($self,$path,$list) = @_; + if ($self->{options}{'caseinsensitive'}) { + $path = lc $path; + } + + while (1) { + if (defined $list->{$path}) { + if (length $list->{$path}) { + return $list->{$path}; + } else { + return 1; + } + } + last unless ($path =~ m/</); + $path =~ s/^<.*?>//; + } + + return 0; +} + +=head2 WORKING WITH ATTRIBUTES + +=over 4 + +=item treat_attributes(@) + +This function handles the translation of the tags' attributes. It receives the tag +without the beginning / end marks, and then it finds the attributes, and it +translates the translatable ones (specified by the module option "attributes"). +This returns a plain string with the translated tag. + +=back + +=cut + +sub treat_attributes { + my ($self,@tag)=@_; + + $tag[0] =~ /^(\S*)(.*)/s; + my $text = $1; + $tag[0] = $2; + + while (@tag) { + my $complete = 1; + + $text .= $self->skip_spaces(\@tag); + if (@tag) { + # Get the attribute's name + $complete = 0; + + $tag[0] =~ /^([^\s=]+)(.*)/s; + my $name = $1; + my $ref = $tag[1]; + $tag[0] = $2; + $text .= $name; + $text .= $self->skip_spaces(\@tag); + if (@tag) { + # Get the '=' + if ($tag[0] =~ /^=(.*)/s) { + $tag[0] = $1; + $text .= "="; + $text .= $self->skip_spaces(\@tag); + if (@tag) { + # Get the value + my $value=""; + $ref=$tag[1]; + my $quot=substr($tag[0],0,1); + if ($quot ne "\"" and $quot ne "'") { + # Unquoted value + $quot=""; + $tag[0] =~ /^(\S+)(.*)/s; + $value = $1; + $tag[0] = $2; + } else { + # Quoted value + $text .= $quot; + $tag[0] =~ /^\Q$quot\E(.*)/s; + $tag[0] = $1; + while ($tag[0] !~ /\Q$quot\E/) { + $value .= $tag[0]; + shift @tag; + shift @tag; + } + $tag[0] =~ /^(.*?)\Q$quot\E(.*)/s; + $value .= $1; + $tag[0] = $2; + } + $complete = 1; + if ($self->tag_in_list($self->get_path.$name,$self->{attributes})) { + $text .= $self->found_string($value, $ref, { type=>"attribute", attribute=>$name }); + } else { + print wrap_ref_mod($ref, "po4a::xml", dgettext("po4a", "Content of attribute %s excluded: %s"), $self->get_path.$name, $value) + if $self->debug(); + $text .= $self->recode_skipped_text($value); + } + $text .= $quot; + } + } + } + + unless ($complete) { + my $ontagerror = $self->{options}{'ontagerror'}; + if ($ontagerror eq "warn") { + warn wrap_ref_mod($ref, "po4a::xml", dgettext ("po4a", "Bad attribute syntax. Continuing...")); + } elsif ($ontagerror ne "silent") { + die wrap_ref_mod($ref, "po4a::xml", dgettext ("po4a", "Bad attribute syntax")); + } + } + } + } + return $text; +} + +# Returns an empty string if the content in the $path should not be +# translated. +# +# Otherwise, returns the set of options for translation: +# w: the content shall be re-wrapped +# W: the content shall not be re-wrapped +# i: the tag shall be inlined +# p: a placeholder shall replace the tag (and its content) +# n: a custom tag +# +# A translatable inline tag in an untranslated tag is treated as a translatable breaking tag. +sub get_translate_options { + my $self = shift; + my $path = shift; + + if (defined $translate_options_cache{$path}) { + return $translate_options_cache{$path}; + } + + my $options = ""; + my $translate = 0; + my $usedefault = 1; + + my $inlist = 0; + my $tag = $self->get_tag_from_list($path, $self->{tags}); + if (defined $tag) { + $inlist = 1; + } + if ($self->{options}{'tagsonly'} eq $inlist) { + $usedefault = 0; + if (defined $tag) { + $options = $tag; + $options =~ s/<.*$//; + } else { + if ($self->{options}{'wrap'}) { + $options = "w"; + } else { + $options = "W"; + } + } + $translate = 1; + } + +# TODO: a less precise set of tags should not override a more precise one + # The tags and tagsonly options are deprecated. + # The translated and untranslated options have an higher priority. + $tag = $self->get_tag_from_list($path, $self->{translated}); + if (defined $tag) { + $usedefault = 0; + $options = $tag; + $options =~ s/<.*$//; + $translate = 1; + } + + if ($translate and $options !~ m/w/i) { + $options .= ($self->{options}{'wrap'})?"w":"W"; + } + + if (not defined $tag) { + $tag = $self->get_tag_from_list($path, $self->{untranslated}); + if (defined $tag) { + $usedefault = 0; + $options = ""; + $translate = 0; + } + } + + $tag = $self->get_tag_from_list($path, $self->{inline}); + if (defined $tag) { + $usedefault = 0; + $options .= "i"; + } else { + $tag = $self->get_tag_from_list($path, $self->{placeholder}); + if (defined $tag) { + $usedefault = 0; + $options .= "p"; + } + } + + $tag = $self->get_tag_from_list($path, $self->{customtag}); + if (defined $tag) { + $usedefault = 0; + $options = "in"; # This erase any other setting + } + + if ($usedefault) { + $options = $self->{options}{'defaulttranslateoption'}; + } + + # A translatable inline tag in an untranslated tag is treated as a + # translatable breaking tag. + if ($options =~ m/i/) { + my $ppath = $path; + $ppath =~ s/<[^>]*>$//; + my $poptions = $self->get_translate_options ($ppath); + if ($poptions eq "") { + $options =~ s/i//; + } + } + + if ($options =~ m/i/ and $self->{options}{'foldattributes'}) { + $options .= "f"; + } + + $translate_options_cache{$path} = $options; + return $options; +} + + +# Return the tag (or biggest set of tags) of a list which matches with the +# given path. +# +# The tag (or set of tags) is returned with its options. +# +# If no tags could match the path, undef is returned. +sub get_tag_from_list ($$$) { + my ($self,$path,$list) = @_; + if ($self->{options}{'caseinsensitive'}) { + $path = lc $path; + } + + while (1) { + if (defined $list->{$path}) { + return $list->{$path}.$path; + } + last unless ($path =~ m/</); + $path =~ s/^<.*?>//; + } + + return undef; +} + + + +sub treat_content { + my $self = shift; + my $blank=""; + # Indicates if the paragraph will have to be translated + my $translate = ""; + + my ($eof,@paragraph)=$self->get_string_until('<',{remove=>1}); + + while (!$eof and !$self->breaking_tag) { + NEXT_TAG: + my @text; + my $type = $self->tag_type; + my $f_extract = $tag_types[$type]->{'f_extract'}; + if ( defined($f_extract) + and $f_extract eq \&tag_extract_comment) { + # Remove the content of the comments + ($eof, @text) = $self->extract_tag($type,1); + $text[$#text-1] .= "\0"; + if ($tag_types[$type]->{'beginning'} eq "!--#") { + $text[0] = "#".$text[0]; + } + push @comments, @text; + } else { + my ($tmpeof, @tag) = $self->extract_tag($type,0); + # Append the found inline tag + ($eof,@text)=$self->get_string_until('>', + {include=>1, + remove=>1, + unquoted=>1}); + # Append or remove the opening/closing tag from + # the tag path + if ($tag_types[$type]->{'end'} eq "") { + if ($tag_types[$type]->{'beginning'} eq "") { + # Opening inline tag + my $cur_tag_name = $self->get_tag_name(@tag); + my $t_opts = $self->get_translate_options($self->get_path($cur_tag_name)); + if ($t_opts =~ m/p/) { + # We enter a new holder. + # Append a <placeholder ...> tag to the current + # paragraph, and save the @paragraph in the + # current holder. + my $last_holder = $save_holders[$#save_holders]; + my $placeholder_str = "<placeholder type=\"".$cur_tag_name."\" id=\"".($#{$last_holder->{'sub_translations'}}+1)."\"/>"; + push @paragraph, ($placeholder_str, $text[1]); + my @saved_paragraph = @paragraph; + + $last_holder->{'paragraph'} = \@saved_paragraph; + + # Then we must push a new holder + my @new_paragraph = (); + my @sub_translations = (); + my %folded_attributes; + my %new_holder = ('paragraph' => \@new_paragraph, + 'open' => $self->join_lines(@text), + 'translation' => "", + 'close' => undef, + 'sub_translations' => \@sub_translations, + 'folded_attributes' => \%folded_attributes); + push @save_holders, \%new_holder; + @text = (); + + # The current @paragraph + # (for the current holder) + # is empty. + @paragraph = (); + } elsif ($t_opts =~ m/f/) { + my $tag_full = $self->join_lines(@text); + my $tag_ref = $text[1]; + if ($tag_full =~ m/^<\s*\S+\s+\S.*>$/s) { + my $holder = $save_holders[$#save_holders]; + my $id = 0; + foreach (keys %{$holder->{folded_attributes}}) { + $id = $_ + 1 if ($_ >= $id); + } + $holder->{folded_attributes}->{$id} = $tag_full; + + @text = ("<$cur_tag_name po4a-id=$id>", $tag_ref); + } + } + unless ($t_opts =~ m/n/) { + push @path, $cur_tag_name; + } + } elsif ($tag_types[$type]->{'beginning'} eq "/") { + # Closing inline tag + + # Check if this is closing the + # last opening tag we detected. + my $test = pop @path; + my $name = $self->get_tag_name(@tag); + if (!defined($test) || + $test ne $name ) { + my $ontagerror = $self->{options}{'ontagerror'}; + if ($ontagerror eq "warn") { + warn wrap_ref_mod($tag[1], "po4a::xml", dgettext("po4a", "Unexpected closing tag </%s> found. The main document may be wrong. Continuing..."), $name); + } elsif ($ontagerror ne "silent") { + die wrap_ref_mod($tag[1], "po4a::xml", dgettext("po4a", "Unexpected closing tag </%s> found. The main document may be wrong."), $name); + } + } + + if ($self->get_translate_options($self->get_path($self->get_tag_name(@tag))) =~ m/p/) { + # This closes the current holder. + + push @path, $self->get_tag_name(@tag); + # Now translate this paragraph if needed. + # This will call pushline and append the + # translation to the current holder's translation. + $self->translate_paragraph(@paragraph); + pop @path; + + # Now that this holder is closed, we can remove + # the holder from the stack. + my $holder = pop @save_holders; + # We need to keep the translation of this holder + my $translation = $holder->{'open'}.$holder->{'translation'}; + $translation .= $self->join_lines(@text); + + @text = (); + + # Then we store the translation in the previous + # holder's sub_translations array + my $previous_holder = $save_holders[$#save_holders]; + push @{$previous_holder->{'sub_translations'}}, $translation; + # We also need to restore the @paragraph array, as + # it was before we encountered the holder. + @paragraph = @{$previous_holder->{'paragraph'}}; + } + } + } + push @paragraph, @text; + } + + # Next tag + ($eof,@text)=$self->get_string_until('<',{remove=>1}); + if ($#text > 0) { + # Check if text (extracted after the inline tag) + # has to be translated + push @paragraph, @text; + } + } + + # This strips the extracted strings + # (only if you don't specify the 'nostrip' option, and if the + # paragraph can be re-wrapped) + $translate = $self->get_translate_options($self->get_path); + if (!$self->{options}{'nostrip'} and $translate !~ m/W/) { + my $clean = 0; + # Clean the beginning + while (!$clean and $#paragraph > 0) { + $paragraph[0] =~ /^(\s*)(.*)/s; + my $match = $1; + if ($paragraph[0] eq $match) { + if ($match ne "") { + $self->pushline($match); + } + shift @paragraph; + shift @paragraph; + } else { + $paragraph[0] = $2; + if ($match ne "") { + $self->pushline($match); + } + $clean = 1; + } + } + $clean = 0; + # Clean the end + while (!$clean and $#paragraph > 0) { + $paragraph[$#paragraph-1] =~ /^(.*?)(\s*)$/s; + my $match = $2; + if ($paragraph[$#paragraph-1] eq $match) { + if ($match ne "") { + $blank = $match.$blank; + } + pop @paragraph; + pop @paragraph; + } else { + $paragraph[$#paragraph-1] = $1; + if ($match ne "") { + $blank = $match.$blank; + } + $clean = 1; + } + } + } + + # Translate the string when needed + # This will either push the translation in the translated document or + # in the current holder translation. + $self->translate_paragraph(@paragraph); + + # Push the trailing blanks + if ($blank ne "") { + $self->pushline($blank); + } + return $eof; +} + +# Translate a @paragraph array of (string, reference). +# The $translate argument indicates if the strings must be translated or +# just pushed +sub translate_paragraph { + my $self = shift; + my @paragraph = @_; + my $translate = $self->get_translate_options($self->get_path); + + while ( (scalar @paragraph) + and ($paragraph[0] =~ m/^\s*\n/s)) { + $self->pushline($paragraph[0]); + shift @paragraph; + shift @paragraph; + } + + my $comments; + while (@comments) { + my ($comment,$eoc); + do { + my ($t,$l) = (shift @comments, shift @comments); + $t =~ s/\n?(\0)?$//; + $eoc = $1; + $comment .= "\n" if defined $comment; + $comment .= $t; + } until ($eoc); + $comments .= "\n" if defined $comments; + $comments .= $comment; + $self->pushline("<!--".$comment."-->\n") if defined $comment; + } + @comments = (); + + if ($self->{options}{'cpp'}) { + my @tmp = @paragraph; + @paragraph = (); + while (@tmp) { + my ($t,$l) = (shift @tmp, shift @tmp); + # #include can be followed by a filename between + # <> brackets. In that case, the argument won't be + # handled in the same call to translate_paragraph. + # Thus do not try to match "include ". + if ($t =~ m/^#[ \t]*(if |endif|undef |include|else|ifdef |ifndef |define )/si) { + if (@paragraph) { + $self->translate_paragraph(@paragraph); + @paragraph = (); + $self->pushline("\n"); + } + $self->pushline($t); + } else { + push @paragraph, ($t,$l); + } + } + } + + my $para = $self->join_lines(@paragraph); + if ( length($para) > 0 ) { + if ($translate ne "") { + # This tag should be translated + $self->pushline($self->found_string( + $para, + $paragraph[1], { + type=>"tag", + tag_options=>$translate, + comments=>$comments + })); + } else { + # Inform that this tag isn't translated in debug mode + print wrap_ref_mod($paragraph[1], "po4a::xml", dgettext ("po4a", "Content of tag %s excluded: %s"), $self->get_path, $para) + if $self->debug(); + $self->pushline($self->recode_skipped_text($para)); + } + } + # Now the paragraph is fully translated. + # If we have all the holders' translation, we can replace the + # placeholders by their translations. + # We must wait to have all the translations because the holders are + # numbered. + { + my $holder = $save_holders[$#save_holders]; + my $translation = $holder->{'translation'}; + + # Count the number of <placeholder ...> in $translation + my $count = 0; + my $str = $translation; + while ( (defined $str) + and ($str =~ m/^.*?<placeholder\s+type="[^"]+"\s+id="(\d+)"\s*\/>(.*)$/s)) { + $count += 1; + $str = $2; + if ($holder->{'sub_translations'}->[$1] =~ m/<placeholder\s+type="[^"]+"\s+id="(\d+)"\s*\/>/s) { + $count = -1; + last; + } + } + + if ( (defined $translation) + and (scalar(@{$holder->{'sub_translations'}}) == $count)) { + # OK, all the holders of the current paragraph are + # closed (and translated). + # Replace them by their translation. + while ($translation =~ m/^(.*?)<placeholder\s+type="[^"]+"\s+id="(\d+)"\s*\/>(.*)$/s) { + # FIXME: we could also check that + # * the holder exists + # * all the holders are used + $translation = $1.$holder->{'sub_translations'}->[$2].$3; + } + # We have our translation + $holder->{'translation'} = $translation; + # And there is no need for any holder in it. + my @sub_translations = (); + $holder->{'sub_translations'} = \@sub_translations; + } + } + +} + + + +=head2 WORKING WITH THE MODULE OPTIONS + +=over 4 + +=item treat_options() + +This function fills the internal structures that contain the tags, attributes +and inline data with the options of the module (specified in the command-line +or in the initialize function). + +=back + +=cut + +sub treat_options { + my $self = shift; + + if ($self->{options}{'caseinsensitive'}) { + $self->{options}{'nodefault'} = lc $self->{options}{'nodefault'}; + $self->{options}{'tags'} = lc $self->{options}{'tags'}; + $self->{options}{'break'} = lc $self->{options}{'break'}; + $self->{options}{'_default_break'} = lc $self->{options}{'_default_break'}; + $self->{options}{'translated'} = lc $self->{options}{'translated'}; + $self->{options}{'_default_translated'} = lc $self->{options}{'_default_translated'}; + $self->{options}{'untranslated'} = lc $self->{options}{'untranslated'}; + $self->{options}{'_default_untranslated'} = lc $self->{options}{'_default_untranslated'}; + $self->{options}{'attributes'} = lc $self->{options}{'attributes'}; + $self->{options}{'_default_attributes'} = lc $self->{options}{'_default_attributes'}; + $self->{options}{'inline'} = lc $self->{options}{'inline'}; + $self->{options}{'_default_inline'} = lc $self->{options}{'_default_inline'}; + $self->{options}{'placeholder'} = lc $self->{options}{'placeholder'}; + $self->{options}{'_default_placeholder'} = lc $self->{options}{'_default_placeholder'}; + $self->{options}{'customtag'} = lc $self->{options}{'customtag'}; + $self->{options}{'_default_customtag'} = lc $self->{options}{'_default_customtag'}; + } + + $self->{options}{'nodefault'} =~ /^\s*(.*)\s*$/s; + my %list_nodefault; + foreach (split(/\s+/s,$1)) { + $list_nodefault{$_} = 1; + } + $self->{nodefault} = \%list_nodefault; + + $self->{options}{'tags'} =~ /^\s*(.*)\s*$/s; + if (length $self->{options}{'tags'}) { + warn wrap_mod("po4a::xml", + dgettext("po4a", + "The '%s' option is deprecated. Please use the translated/untranslated and/or break/inline/placeholder categories."), "tags"); + } + foreach (split(/\s+/s,$1)) { + $_ =~ m/^(.*?)(<.*)$/; + $self->{tags}->{$2} = $1 || ""; + } + + if ($self->{options}{'tagsonly'}) { + warn wrap_mod("po4a::xml", + dgettext("po4a", + "The '%s' option is deprecated. Please use the translated/untranslated and/or break/inline/placeholder categories."), "tagsonly"); + } + + $self->{options}{'break'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{break}->{$2} = $1 || ""; + } + $self->{options}{'_default_break'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{break}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{break}->{$2}; + } + + $self->{options}{'translated'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{translated}->{$2} = $1 || ""; + } + $self->{options}{'_default_translated'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{translated}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{translated}->{$2}; + } + + $self->{options}{'untranslated'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{untranslated}->{$2} = $1 || ""; + } + $self->{options}{'_default_untranslated'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{untranslated}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{untranslated}->{$2}; + } + + $self->{options}{'attributes'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + if ($tag =~ m/^(.*?)(<.*)$/) { + $self->{attributes}->{$2} = $1 || ""; + } else { + $self->{attributes}->{$tag} = ""; + } + } + $self->{options}{'_default_attributes'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + if ($tag =~ m/^(.*?)(<.*)$/) { + $self->{attributes}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{attributes}->{$2}; + } else { + $self->{attributes}->{$tag} = "" + unless $list_nodefault{$tag} + or defined $self->{attributes}->{$tag}; + } + } + + $self->{options}{'inline'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{inline}->{$2} = $1 || ""; + } + $self->{options}{'_default_inline'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{inline}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{inline}->{$2}; + } + + $self->{options}{'placeholder'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{placeholder}->{$2} = $1 || ""; + } + $self->{options}{'_default_placeholder'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{placeholder}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{placeholder}->{$2}; + } + + $self->{options}{'customtag'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{customtag}->{$2} = $1 || ""; + } + $self->{options}{'_default_customtag'} =~ /^\s*(.*)\s*$/s; + foreach my $tag (split(/\s+/s,$1)) { + $tag =~ m/^(.*?)(<.*)$/; + $self->{customtag}->{$2} = $1 || "" + unless $list_nodefault{$2} + or defined $self->{customtag}->{$2}; + } + + # There should be no translated and untranslated tags + foreach my $tag (keys %{$self->{translated}}) { + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "translated", "untranslated") + if defined $self->{untranslated}->{$tag}; + } + # There should be no inline, break, placeholder, and customtag tags + foreach my $tag (keys %{$self->{inline}}) { + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "inline", "break") + if defined $self->{break}->{$tag}; + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "inline", "placeholder") + if defined $self->{placeholder}->{$tag}; + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "inline", "customtag") + if defined $self->{customtag}->{$tag}; + } + foreach my $tag (keys %{$self->{break}}) { + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "break", "placeholder") + if defined $self->{placeholder}->{$tag}; + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "break", "customtag") + if defined $self->{customtag}->{$tag}; + } + foreach my $tag (keys %{$self->{placeholder}}) { + die wrap_mod("po4a::xml", + dgettext("po4a", + "Tag '%s' both in the %s and %s categories."), $tag, "placeholder", "customtag") + if defined $self->{customtag}->{$tag}; + } +} + +=head2 GETTING TEXT FROM THE INPUT DOCUMENT + +=over + +=item get_string_until($%) + +This function returns an array with the lines (and references) from the input +document until it finds the first argument. The second argument is an options +hash. Value 0 means disabled (the default) and 1, enabled. + +The valid options are: + +=over 4 + +=item B<include> + +This makes the returned array to contain the searched text + +=item B<remove> + +This removes the returned stream from the input + +=item B<unquoted> + +This ensures that the searched text is outside any quotes + +=back + +=cut + +sub get_string_until { + my ($self,$search) = (shift,shift); + my $options = shift; + my ($include,$remove,$unquoted, $regex) = (0,0,0,0); + + if (defined($options->{include})) { $include = $options->{include}; } + if (defined($options->{remove})) { $remove = $options->{remove}; } + if (defined($options->{unquoted})) { $unquoted = $options->{unquoted}; } + if (defined($options->{regex})) { $regex = $options->{regex}; } + + my ($line,$ref) = $self->shiftline(); + my (@text,$paragraph); + my ($eof,$found) = (0,0); + + $search = "\Q$search\E" unless $regex; + while (defined($line) and !$found) { + push @text, ($line,$ref); + $paragraph .= $line; + if ($unquoted) { + if ( $paragraph =~ /^((\".*?\")|(\'.*?\')|[^\"\'])*$search/s ) { + $found = 1; + } + } else { + if ( $paragraph =~ /$search/s ) { + $found = 1; + } + } + if (!$found) { + ($line,$ref)=$self->shiftline(); + } + } + + if (!defined($line)) { $eof = 1; } + + if ( $found ) { + $line = ""; + if($unquoted) { + $paragraph =~ /^(?:(?:\".*?\")|(?:\'.*?\')|[^\"\'])*?$search(.*)$/s; + $line = $1; + $text[$#text-1] =~ s/\Q$line\E$//s; + } else { + $paragraph =~ /$search(.*)$/s; + $line = $1; + $text[$#text-1] =~ s/\Q$line\E$//s; + } + if(!$include) { + $text[$#text-1] =~ /^(.*)($search.*)$/s; + $text[$#text-1] = $1; + $line = $2.$line; + } + if (defined($line) and ($line ne "")) { + $self->unshiftline ($line,$text[$#text]); + } + } + if (!$remove) { + $self->unshiftline (@text); + } + + #If we get to the end of the file, we return the whole paragraph + return ($eof,@text); +} + +=item skip_spaces(\@) + +This function receives as argument the reference to a paragraph (in the format +returned by get_string_until), skips his heading spaces and returns them as +a simple string. + +=cut + +sub skip_spaces { + my ($self,$pstring)=@_; + my $space=""; + + while (@$pstring and (@$pstring[0] =~ /^(\s+)(.*)$/s or @$pstring[0] eq "")) { + if (@$pstring[0] ne "") { + $space .= $1; + @$pstring[0] = $2; + } + + if (@$pstring[0] eq "") { + shift @$pstring; + shift @$pstring; + } + } + return $space; +} + +=item join_lines(@) + +This function returns a simple string with the text from the argument array +(discarding the references). + +=cut + +sub join_lines { + my ($self,@lines)=@_; + my ($line,$ref); + my $text = ""; + while ($#lines > 0) { + ($line,$ref) = (shift @lines,shift @lines); + $text .= $line; + } + return $text; +} + +=back + +=head1 STATUS OF THIS MODULE + +This module can translate tags and attributes. + +=head1 TODO LIST + +DOCTYPE (ENTITIES) + +There is a minimal support for the translation of entities. They are +translated as a whole, and tags are not taken into account. Multilines +entities are not supported and entities are always rewrapped during the +translation. + +MODIFY TAG TYPES FROM INHERITED MODULES +(move the tag_types structure inside the $self hash?) + +=head1 SEE ALSO + +L<Locale::Po4a::TransTractor(3pm)|Locale::Po4a::TransTractor>, +L<po4a(7)|po4a.7> + +=head1 AUTHORS + + Jordi Vilalta <jvprat@gmail.com> + Nicolas François <nicolas.francois@centraliens.net> + +=head1 COPYRIGHT AND LICENSE + + Copyright (c) 2004 by Jordi Vilalta <jvprat@gmail.com> + Copyright (c) 2008-2009 by Nicolas François <nicolas.francois@centraliens.net> + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut + +1; Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Chooser.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Chooser.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Chooser.pm (revision 2840) @@ -0,0 +1,151 @@ +# Locale::Po4a::Chooser -- Manage po4a modules +# +# This program is free software; you may redistribute it and/or modify it +# under the terms of GPL (see COPYING). +# +# This module converts POD to PO file, so that it becomes possible to +# translate POD formatted documentation. See gettext documentation for +# more info about PO files. + +############################################################################ +# Modules and declarations +############################################################################ + + +package Locale::Po4a::Chooser; + +use 5.006; +use strict; +use warnings; +use Locale::Po4a::Common; + +sub new { + my ($module)=shift; + my (%options)=@_; + + die wrap_mod("po4a::chooser", gettext("Need to provide a module name")) + unless defined $module; + + my $modname; + if ($module eq 'kernelhelp') { + $modname = 'KernelHelp'; + } elsif ($module eq 'newsdebian') { + $modname = 'NewsDebian'; + } elsif ($module eq 'latex') { + $modname = 'LaTeX'; + } elsif ($module eq 'bibtex') { + $modname = 'BibTex'; + } elsif ($module eq 'tex') { + $modname = 'TeX'; + } else { + $modname = ucfirst($module); + } + if (! UNIVERSAL::can("Locale::Po4a::$modname", 'new')) { + eval qq{use Locale::Po4a::$modname}; + if ($@) { + my $error=$@; + warn wrap_msg(gettext("Unknown format type: %s."), $module); + warn wrap_mod("po4a::chooser", + gettext("Module loading error: %s"), $error) + if defined $options{'verbose'} && $options{'verbose'} > 0; + list(1); + } + } + return "Locale::Po4a::$modname"->new(%options); +} + +sub list { + warn wrap_msg(gettext("List of valid formats:") +# ."\n - ".gettext("bibtex: BibTex bibliography format.") + ."\n - ".gettext("dia: uncompressed Dia diagrams.") + ."\n - ".gettext("docbook: DocBook XML.") + ."\n - ".gettext("guide: Gentoo Linux's XML documentation format.") +# ."\n - ".gettext("html: HTML documents (EXPERIMENTAL).") + ."\n - ".gettext("ini: INI format.") + ."\n - ".gettext("kernelhelp: Help messages of each kernel compilation option.") + ."\n - ".gettext("latex: LaTeX format.") + ."\n - ".gettext("man: Good old manual page format.") + ."\n - ".gettext("pod: Perl Online Documentation format.") + ."\n - ".gettext("sgml: either DebianDoc or DocBook DTD.") + ."\n - ".gettext("texinfo: The info page format.") + ."\n - ".gettext("tex: generic TeX documents (see also latex).") + ."\n - ".gettext("text: simple text document.") + ."\n - ".gettext("prop: Java property and OSX/NeXTSTEP strings files.") + ."\n - ".gettext("wml: WML documents.") + ."\n - ".gettext("xhtml: XHTML documents.") + ."\n - ".gettext("xml: generic XML documents (see also docbook).") + ); + exit shift; +} +############################################################################## +# Module return value and documentation +############################################################################## + +1; +__END__ + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Chooser - manage po4a modules + +=head1 DESCRIPTION + +Locale::Po4a::Chooser is a module to manage po4a modules. Previously, all po4a +binaries used to know all po4a modules (pod, man, sgml, etc). This made the +addition of a new module boring, to make sure the documentation is synchronized +in all modules, and that each of them can access the new module. + +Now, you just have to call the Locale::Po4a::Chooser::new() function, +passing the name of module as argument. + +You also have the Locale::Po4a::Chooser::list() function which lists the +available formats and exits on the value passed as argument. + +=head1 SEE ALSO + +=over 4 + +=item About po4a: + +L<Locale::Po4a::Po(3pm)>, +L<Locale::Po4a::TransTractor(3pm)>, +L<po4a(7)|po4a.7> + +=item About modules: + +L<Locale::Po4a::Dia(3pm)>, +L<Locale::Po4a::Docbook(3pm)>, +L<Locale::Po4a::Guide(3pm)>, +L<Locale::Po4a::Halibut(3pm)>, +L<Locale::Po4a::Ini(3pm)>, +L<Locale::Po4a::KernelHelp(3pm)>, +L<Locale::Po4a::LaTeX(3pm)>, +L<Locale::Po4a::Man(3pm)>, +L<Locale::Po4a::Pod(3pm)>, +L<Locale::Po4a::Sgml(3pm)>, +L<Locale::Po4a::TeX(3pm)>, +L<Locale::Po4a::Texinfo(3pm)>, +L<Locale::Po4a::Text(3pm)>, +L<Locale::Po4a::Prop(3pm)>, +L<Locale::Po4a::Wml(3pm)>. +L<Locale::Po4a::Xhtml(3pm)>, +L<Locale::Po4a::Xml(3pm)>, +L<Locale::Po4a::Wml(3pm)>. + +=back + +=head1 AUTHORS + + Denis Barbier <barbier@linuxfr.org> + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + +Copyright 2002,2003,2004,2005 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). + +=cut Index: branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Sgml.pm =================================================================== --- branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Sgml.pm (revision 0) +++ branches/Bungo/package/bin/po4a/lib/Locale/Po4a/Sgml.pm (revision 2840) @@ -0,0 +1,1273 @@ +#!/usr/bin/perl -w + +# Po4a::Sgml.pm +# +# extract and translate translatable strings from an sgml based document. +# +# This code is an adapted version of sgmlspl (SGML postprocessor for the +# SGMLS and NSGMLS parsers) which was: +# +# Copyright (c) 1995 by David Megginson <dmeggins@aix1.uottawa.ca> +# +# The adaptation for po4a was done by Denis Barbier <barbier@linuxfr.org>, +# Martin Quinson (mquinson#debian.org) and others. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +######################################################################## + +=encoding UTF-8 + +=head1 NAME + +Locale::Po4a::Sgml - convert SGML documents from/to PO files + +=head1 DESCRIPTION + +The po4a (PO for anything) project goal is to ease translations (and more +interestingly, the maintenance of translations) using gettext tools on +areas where they were not expected like documentation. + +Locale::Po4a::Sgml is a module to help the translation of documentation in +the SGML format into other [human] languages. + +This module uses B<nsgmls> to parse the SGML files. Make sure it is +installed. +Also make sure that the DTD of the SGML files are installed in the system. + +=head1 OPTIONS ACCEPTED BY THIS MODULE + +=over 4 + +=item B<debug> + +Space separated list of keywords indicating which part you want to debug. Possible values are: tag, generic, entities and refs. + +=item B<verbose> + +Give more information about what's going on. + +=item B<translate> + +Space separated list of extra tags (beside the DTD provided ones) whose +content should form an extra msgid. + +=item B<section> + +Space separated list of extra tags (beside the DTD provided ones) +containing other tags, some of them being of category B<translate>. + +=item B<indent> + +Space separated list of tags which increase the indentation level. + +=item B<verbatim> + +The layout within those tags should not be changed. The paragraph won't get +wrapped, and no extra indentation space or new line will be added for +cosmetic purpose. + +=item B<empty> + +Tags not needing to be closed. + +=item B<ignore> + +Tags ignored and considered as plain char data by po4a. That is to say that +they can be part of an msgid. For example, E<lt>bE<gt> is a good candidate +for this category since putting it in the translate section would create +msgids not being whole sentences, which is bad. + +=item B<attributes> + +A space separated list of attributes that need to be translated. You can +specify the attributes by their name (for example, "lang"), but you can also +prefix it with a tag hierarchy, to specify that this attribute will only be +translated when it is into the specified tag. For example: +E<lt>bbbE<gt>E<lt>aaaE<gt>lang specifies that the lang attribute will only be +translated if it is in an E<lt>aaaE<gt> tag, which is in a E<lt>bbbE<gt> tag. +The tag names are actually regular expressions so you can also write things +like E<lt>aaa|bbbbE<gt>lang to only translate lang attributes that are in +an E<lt>aaaE<gt> or a E<lt>bbbE<gt> tag. + +=item B<qualify> + +A space separated list of attributes for which the translation must be +qualified by the attribute name. Note that this setting automatically adds the +given attribute into the 'attributes' list too. + +=item B<force> + +Proceed even if the DTD is unknown or if nsgmls finds errors in the input +file. + +=item B<include-all> + +By default, msgids containing only one entity (like '&version;') are skipped +for the translator comfort. Activating this option prevents this +optimisation. It can be useful if the document contains a construction like +"<title>Á", even if I doubt such things to ever happen... + +=item B + +Space separated list of entities that won't be inlined. +Use this option with caution: it may cause nsgmls (used internally) to add +tags and render the output document invalid. + +=back + +=head1 STATUS OF THIS MODULE + +The result is perfect. I.e., the generated documents are exactly the +same. But there are still some problems: + +=over 2 + +=item * + +The error output of nsgmls is redirected to /dev/null, which is clearly +bad. I don't know how to avoid that. + +The problem is that I have to "protect" the conditional inclusions (i.e. the +C! [ %foo [> and C<]]E> stuff) from nsgmls. Otherwise +nsgmls eats them, and I don't know how to restore them in the final +document. To prevent that, I rewrite them to C<{PO4A-beg-foo}> and +C<{PO4A-end}>. + +The problem with this is that the C<{PO4A-end}> and such I add are valid in +the document (not in a EpE tag or so). + +Everything works well with nsgmls's output redirected that way, but it will +prevent us from detecting that the document is badly formatted. + +=item * + +It does work only with the DebianDoc and DocBook DTD. Adding support for a +new DTD should be very easy. The mechanism is the same for every DTD, you just +have to give a list of the existing tags and some of their characteristics. + +I agree, this needs some more documentation, but it is still considered as +beta, and I hate to document stuff which may/will change. + +=item * + +Warning, support for DTDs is quite experimental. I did not read any +reference manual to find the definition of every tag. I did add tag +definition to the module 'till it works for some documents I found on the +net. If your document use more tags than mine, it won't work. But as I said +above, fixing that should be quite easy. + +I did test DocBook against the SAG (System Administrator Guide) only, but +this document is quite big, and should use most of the DocBook +specificities. + +For DebianDoc, I tested some of the manuals from the DDP, but not all yet. + +=item * + +In case of file inclusion, string reference of messages in PO files +(i.e. lines like C<#: en/titletoc.sgml:9460>) will be wrong. + +This is because I preprocess the file to protect the conditional inclusion +(i.e. the C! [ %foo [> and C<]]E> stuff) and some entities (like +&version;) from nsgmls because I want them verbatim to the generated +document. For that, I make a temp copy of the input file and do all the +changes I want to this before passing it to nsgmls for parsing. + +So that it works, I replace the entities asking for a file inclusion by the +content of the given file (so that I can protect what needs to be in a subfile +also). But nothing is done so far to correct the references (i.e., filename +and line number) afterward. I'm not sure what the best thing to do is. + +=back + +=cut + +package Locale::Po4a::Sgml; + +use 5.006; +use strict; +use warnings; + + +require Exporter; +use vars qw(@ISA @EXPORT); +@ISA = qw(Locale::Po4a::TransTractor); +@EXPORT = qw(); + +use Locale::Po4a::TransTractor; +use Locale::Po4a::Common; + +eval qq{use SGMLS}; +if ($@) { + die wrap_mod("po4a::sgml", dgettext("po4a","The needed module SGMLS.pm was not found and needs to be installed. It can be found on the CPAN, in package libsgmls-perl on debian, etc.")); +} + +use File::Temp; + +my %debug=('tag' => 0, + 'generic' => 0, + 'entities' => 0, + 'refs' => 0, + 'nsgmls' => 0); + +my $xmlprolog = undef; # the '' line if existing + +sub initialize { + my $self = shift; + my %options = @_; + + $self->{options}{'translate'}=''; + $self->{options}{'section'}=''; + $self->{options}{'indent'}=''; + $self->{options}{'empty'}=''; + $self->{options}{'verbatim'}=''; + $self->{options}{'ignore'}=''; + $self->{options}{'ignore-inclusion'}=''; + + $self->{options}{'include-all'}=''; + + $self->{options}{'force'}=''; + + $self->{options}{'verbose'}=''; + $self->{options}{'debug'}=''; + + foreach my $opt (keys %options) { + if ($options{$opt}) { + die wrap_mod("po4a::sgml", dgettext ("po4a", "Unknown option: %s"), $opt) unless exists $self->{options}{$opt}; + $self->{options}{$opt} = $options{$opt}; + } + } + if ($options{'debug'}) { + foreach (split /\s+/, $options{'debug'}) { + $debug{$_} = 1; + } + } +} + +sub read { + my ($self,$filename)=@_; + + push @{$self->{DOCPOD}{infile}}, $filename; + $self->Locale::Po4a::TransTractor::read($filename); +} + +sub parse { + my $self=shift; + map {$self->parse_file($_)} @{$self->{DOCPOD}{infile}}; +} + +# +# Filter out some uninteresting strings for translation +# +sub translate { + my ($self)=(shift); + my ($string,$ref,$type)=(shift,shift,shift); + my (%options)=@_; + + # don't translate entries composed of one entity + if ( (($string =~ /^&[^;]*;$/) || ($options{'wrap'} && $string =~ /^\s*&[^;]*;\s*$/)) + && !($self->{options}{'include-all'}) ){ + warn wrap_mod("po4a::sgml", dgettext("po4a", "msgid skipped to help translators (contains only an entity)"), $string) + unless $self->verbose() <= 0; + return $string.($options{'wrap'}?"\n":""); + } + # don't translate entries composed of tags only + if ( $string =~ /^(((<[^>]*>)|\s)*)$/ + && !($self->{options}{'include-all'}) ) { + warn wrap_mod("po4a::sgml", dgettext("po4a", "msgid skipped to help translators (contains only tags)"), $string) + unless $self->verbose() <= 0; + return $string.($options{'wrap'}?"\n":""); + } + + # don't translate entries composed of marked section tags only + if ( ($string =~ /^(?:|\s)*$/) + && !($self->{options}{'include-all'})) { + warn wrap_mod("po4a::sgml", dgettext("po4a", "msgid skipped to ". + "help translators (contains only opening or closing ". + "tags of marked sections)"), $string) + unless $self->verbose() <= 0; + return $string.($options{'wrap'}?"\n":""); + } + + $string = $self->SUPER::translate($string,$ref,$type,%options); + + $string = $self->post_trans($string,$ref,$type); + + return $string; +} + +sub post_trans { + my ($self,$str,$ref,$type)=@_; + + # Change ascii non-breaking space to an   + my $nbs_out = "\xA0"; + my $enc_length = Encode::from_to($nbs_out, "latin1", + $self->get_out_charset); + $str =~ s/\Q$nbs_out/ /g if defined $enc_length; + + return $str; +} + +# +# Make sure our cruft is removed from the file +# +sub pushline { + my ($self,$line)=@_; + $line =~ s/{PO4A-amp}/&/g; + $self->SUPER::pushline($line); +} + +sub set_tags_kind { + my $self=shift; + my (%kinds)=@_; + + foreach (qw(translate empty section verbatim ignore attributes qualify)) { + $self->{SGML}->{k}{$_} = $self->{options}{$_} ? $self->{options}{$_}.' ' : ''; + # Remove the default behavior for the tags defined with the + # options. + foreach my $k (keys %kinds) { + foreach my $t (split(" ", $self->{SGML}->{k}{$_})) { + $kinds{$k} =~ s/\b$t\b//; + } + } + } + + foreach (keys %kinds) { + die "po4a::sgml: internal error: set_tags_kind called with unrecognized arg $_" + if ($_ !~ /^(translate|empty|verbatim|ignore|indent|attributes|qualify)$/); + + $self->{SGML}->{k}{$_} .= $kinds{$_}; + } +} + +# +# Do the actual work, using the SGMLS package and settings done elsewhere. +# +sub parse_file { + my ($self,$mastername)=@_; + my ($prolog); + + # Rewrite the file to: + # - protect optional inclusion marker (i.e. "") + # - protect entities from expansion (ie "&release;") + my $origfile=""; + my $i=0; + while (defined(@{$self->{TT}{doc_in}}) && $i < @{$self->{TT}{doc_in}}) { + $origfile .= ${$self->{TT}{doc_in}}[$i]; + $i+=2; + } + + unless ($self->{options}{'force'}) { + # Detect if we can find the DTD + my ($tmpfh,$tmpfile)=File::Temp::tempfile("po4a-XXXX", + SUFFIX => ".sgml", + DIR => "/tmp", + UNLINK => 0); + print $tmpfh $origfile; + close $tmpfh + or die wrap_mod("po4a::sgml", + dgettext("po4a", "Can't close tempfile: %s"), $!); + if (system("nsgmls -p < $tmpfile")) { + unlink ($tmpfile); + die wrap_mod("po4a::sgml", + dgettext("po4a", "Error while running nsgmls -p. ". + "Please check if nsgmls and the ". + "DTD are installed.")); + } + unlink ($tmpfile); + } + # Detect the XML pre-prolog + if ($origfile =~ s/^(\s*<\?xml[^?]*\?>)//) { + warn wrap_mod("po4a::sgml", dgettext("po4a", + "Trying to handle a XML document as a SGML one. ". + "Feel lucky if it works, help us implementing a proper XML backend if it does not."), $mastername) + unless $self->verbose() <= 0; + $xmlprolog=$1; + } + # Get the prolog + { + $prolog=$origfile; + my $lvl; # number of '<' seen without matching '>' + my $pos = 0; # where in the document (in chars) while detecting prolog boundaries + + unless ($prolog =~ s/^(.*' or '<' in them. + if ($origfile =~ m/^.{$pos}?()/s) { + print "Found a comment in the prolog: $1\n" if ($debug{'generic'}); + $pos += length($1); + # take care of the line numbers + my @a = split(/\n/,$1); + shift @a; # nb line - 1 + while (defined(shift @a)) { + $prolog .= "\n"; + } + next; + } + # Search the closing '>' + my ($c)=substr($origfile,$pos,1); + $lvl++ if ($c eq '<'); + $lvl-- if ($c eq '>'); + $prolog = "$prolog$c"; + $pos++; + } + } + + # Add the definition of new tags that will be used for the + # conditionnal inclusions + if ($origfile =~ /^.*]*\[/is) { + $origfile =~ s/^(.*]*\[)/$1 /is; + } + + print STDERR "PROLOG=$prolog\n------------\n" if ($debug{'generic'}); + + # Configure the tags for this dtd + if ($prolog =~ /debiandoc/i) { + $self->set_tags_kind("translate" => "author version abstract title". + "date copyrightsummary heading p ". + "example tag title", + "empty" => "date ref manref url toc", + "verbatim" => "example", + "ignore" => "package prgn file tt em var ". + "name email footnote po4aend po4abeg ". + "strong ftpsite ftppath qref", + "indent" => "appendix ". + "book ". + "chapt copyright ". + "debiandoc ". + "enumlist ". + "item ". + "list ". + "sect sect1 sect2 sect3 sect4 ". + "tag taglist titlepag toc"); + + } elsif ($prolog =~ /docbook/i) { + $self->set_tags_kind("translate" => "abbrev appendixinfo artheader attribution ". + "biblioentry biblioset ". + "chapterinfo collab collabname confdates confgroup conftitle ". + "date ". + "edition editor entry example ". + "figure ". + "glosssee glossseealso glossterm ". + "holder ". + "member msgaud msglevel msgorig ". + "orgdiv orgname othername ". + "pagenums para phrase pubdate publishername primary ". + "refclass refdescriptor refentrytitle refmiscinfo refname refpurpose releaseinfo remark revnumber revremark ". + "screeninfo seg secondary see seealso segtitle simpara substeps subtitle synopfragmentref synopsis ". + "term tertiary title titleabbrev ". + "contrib epigraph", + "empty" => "audiodata colspec graphic imagedata textdata sbr spanspec videodata xref", + "indent" => "abstract answer appendix article articleinfo audioobject author authorgroup ". + "bibliodiv bibliography blockquote blockinfo book bookinfo bridgehead ". + "callout calloutlist caption caution chapter copyright ". + "dedication docinfo ". + "entry ". + "formalpara ". + "glossary glossdef glossdiv glossentry glosslist group ". + "imageobject important index indexterm informaltable itemizedlist ". + "keyword keywordset ". + "legalnotice listitem lot ". + "mediaobject msg msgentry msginfo msgexplan msgmain msgrel msgsub msgtext ". + "note ". + "objectinfo orderedlist ". + "part partintro preface procedure publisher ". + "qandadiv qandaentry qandaset question ". + "reference refentry refentryinfo refmeta refnamediv refsect1 refsect1info refsect2 refsect2info refsect3 refsect3info refsection refsectioninfo refsynopsisdiv refsynopsisdivinfo revision revdescription row ". + "screenshot sect1 sect1info sect2 sect2info sect3 sect3info sect4 sect4info sect5 sect5info section sectioninfo seglistitem segmentedlist set setindex setinfo shortcut simplelist simplemsgentry simplesect step synopfragment ". + "table tbody textobject tgroup thead tip toc ". + "variablelist varlistentry videoobject ". + "warning", + "verbatim" => "address cmdsynopsis holder literallayout programlisting ". + "refentrytitle refname refpurpose screen term title", + "ignore" => "acronym action affiliation anchor application arg author authorinitials ". + "city citation citerefentry citetitle classname co command computeroutput constant corpauthor country ". + "database po4abeg po4aend ". + "email emphasis envar errorcode errorname errortext errortype exceptionname ". + "filename firstname firstterm footnote footnoteref foreignphrase function ". + "glossterm guibutton guiicon guilabel guimenu guimenuitem guisubmenu ". + "hardware ". + "indexterm informalexample inlineequation inlinegraphic inlinemediaobject interface interfacename isbn ". + "keycap keycode keycombo keysym ". + "link lineannotation literal ". + "manvolnum markup medialabel menuchoice methodname modespec mousebutton ". + "nonterminal ". + "olink ooclass ooexception oointerface option optional othercredit ". + "parameter personname phrase productname productnumber prompt property pubsnumber ". + "quote ". + "remark replaceable returnvalue revhistory ". + "sgmltag sidebar structfield structname subscript superscript surname symbol systemitem ". + "token trademark type ". + "ulink userinput ". + "varname volumenum ". + "wordasword ". + "xref ". + "year", + "attributes" =>"<(article|book)>lang"); + + } else { + if ($self->{options}{'force'}) { + warn wrap_mod("po4a::sgml", dgettext("po4a", "DTD of this file is unknown, but proceeding as requested.")); + $self->set_tags_kind(); + } else { + die wrap_mod("po4a::sgml", dgettext("po4a", + "DTD of this file is unknown. (supported: DebianDoc, DocBook). The prolog follows:")."\n$prolog"); + } + } + + # Hash of the file entities that won't be included + my %ignored_inclusion = (); + foreach (split / /,$self->{options}{'ignore-inclusion'}) { + $ignored_inclusion{$_} = 1; + } + + # Prepare the reference indirection stuff + my @refs; + my $length = ($origfile =~ tr/\n/\n/); + print "XX Prepare reference indirection stuff\n" if $debug{'refs'}; + for (my $i=1; $i<=$length; $i++) { + push @refs,"$mastername:$i"; + print "$mastername:$i\n" if $debug{'refs'}; + } + + # protect the conditional inclusions in the file + $origfile =~ s//g; # cond. incl. starts + $origfile =~ s/\]\]>//g; # cond. incl. end + + # Remove )(.*)$/s) + { + my ($begin,$ignored,$end) = ($1, $2, $3); + my @begin = split(/\n/, $begin); + my @ignored = split(/\n/, $ignored); + my $pre = scalar @begin; + my $len = (scalar @ignored) -1; + $pre++ if ($begin =~ /\n$/s); + $len++ if ($end =~ /^\n/s); + # remove the references of the ignored lines + splice @refs, $pre+1, $len-1; + # remove the lines + $tmp1 = $begin.$end; + } + $origfile = $tmp1; + # The <, >, and & in a CDATA must be escaped because they do not + # correspond to tags or entities delimiters. + $tmp1 = $origfile; + $origfile = ""; + while ($tmp1 =~ m/^(.*?{PO4A-beg-\s*(?:CDATA|RCDATA)\s*})(.+?)(.*)$/s) { + my ($begin, $tmp) = ($1, $2); + $tmp1 = $3; + $tmp =~ s//{PO4A-gt}/gs; + $tmp =~ s/&/{PO4A-amp}/gs; + $origfile .= $begin.$tmp; + } + $origfile .= $tmp1; + + # Deal with the %entities; in the prolog. God damn it, this code is gross! + # Try hard not to change the number of lines to not fuck up the references + my %prologentincl; + my $moretodo=1; + PROLOGENTITY: while ($moretodo) { # non trivial loop to deal with recursive inclusion + $moretodo = 0; + # Unprotect not yet defined inclusions + $prolog =~ s/{PO4A-percent}/%/sg; + print STDERR "prolog=>>>>$prolog<<<<\n" + if ($debug{'entities'}); + while ($prolog =~ /(.*?)"]*)"\s*>(.*)$/is) { #})"{ (Stupid editor) + print STDERR "Seen the definition entity of prolog inclusion '$2' (=$3)\n" + if ($debug{'entities'}); + # Preload the content of the entity. + my $key = $2; + my $filename=$3; + my $origfilename = $filename; + my ($begin, $end) = ($1, $4); + if ($filename !~ m%^/% && $mastername =~ m%/%) { + my $dir=$mastername; + $dir =~ s%/[^/]*$%%; + $filename="$dir/$filename"; + # origfile also needs to be fixed otherwise nsgmls won't + # find the file. + $origfile =~ s/()/$1$filename$2/gsi; + } + if (defined $ignored_inclusion{$key} or !-e $filename) { + # We won't expand this entity. + # And we avoid nsgmls to do so. + $prolog = "$begin$end"; + } else { + $prolog = $begin.$end; + (-e $filename && open IN,"<$filename") || + die wrap_mod("po4a::sgml", + dgettext("po4a", + "Can't open %s (content of entity %s%s;): %s"), + $filename, '%', $key, $!); + local $/ = undef; + $prologentincl{$key} = ; + close IN; + print STDERR "Content of \%$key; is $filename (". + ($prologentincl{$key} =~ tr/\n/\n/). + " lines long)\n" + if ($debug{'entities'}); + print STDERR "content: ".$prologentincl{$key}."\n" + if ($debug{'entities'}); + $moretodo = 1; + next PROLOGENTITY; + } + } + while ($prolog =~ /(.*?)"]*)"\s*>(.*)$/is) { #})"{ (Stupid editor) + print STDERR "Seen the definition entity of prolog definition '$2' (=$3)\n" + if ($debug{'entities'}); + # Preload the content of the entity. + my $key = $2; + $prolog = $1.$4; + $prologentincl{$key} = $3; + print STDERR "content: ".$prologentincl{$key}."\n" + if ($debug{'entities'}); + $moretodo = 1; + next PROLOGENTITY; + } + while ($prolog =~ /^(.*?)%([^;\s]*);(.*)$/s) { + my ($pre,$ent,$post) = ($1,$2,$3); + # Yeah, right, the content of the entity can be defined in a not yet loaded entity + # It's easy to build a weird case where all that shit collapses poorly. But why the + # hell are you using those strange constructs in your document, damn it? + print STDERR "Seen prolog inclusion $ent\n" if ($debug{'entities'}); + if (defined ($prologentincl{$ent})) { + $prolog = $pre.$prologentincl{$ent}.$post; + print STDERR "Change \%$ent; to its content in the prolog\n" + if $debug{'entities'}; + $moretodo = 1; + } else { + # AAAARGH stupid document using %bla; and having then defined in another inclusion! + # Protect it for this pass, and unprotect it on next one + print STDERR "entity $ent not defined yet ?!\n" + if $debug{'entities'}; + $prolog = "$pre".'{PO4A-percent}'."$ent;$post"; + } + } + } + $prolog =~ s///g; + # Unprotect undefined inclusions, and die of them + $prolog =~ s/{PO4A-percent}/%/sg; + if ($prolog =~ /%([^;\s]*);/) { + die wrap_mod("po4a::sgml", + dgettext("po4a", + "unrecognized prolog inclusion entity: %%%s;"), + $1) + unless ($ignored_inclusion{$1}); + } + # Protect &entities; (all but the ones asking for a file inclusion) + # search the file inclusion entities + my %entincl; + my $searchprolog=$prolog; + while ($searchprolog =~ /(.*?)"]*)"\s*>(.*)$/is) { #})"{ + print STDERR "Seen the entity of inclusion $2 (=$3)\n" + if ($debug{'entities'}); + my $key = $2; + my $filename = $3; + my $origfilename = $filename; + $searchprolog = $4; + if ($filename !~ m%^/% && $mastername =~ m%/%) { + my $dir=$mastername; + $dir =~ s%/[^/]*$%%; + $filename="$dir/$filename"; + # origfile also needs to be fixed otherwise nsgmls won't find + # the file. + $origfile =~ s/()/$1$filename$2/gsi; + } + if ((not defined $ignored_inclusion{$2}) and (-e $filename)) { + $entincl{$key}{'filename'}=$filename; + # Preload the content of the entity + (-e $filename && open IN,"<$filename") || + die wrap_mod("po4a::sgml", + dgettext("po4a", + "Can't open %s (content of entity %s%s;): %s"), + $filename, '&', $key, $!); + local $/ = undef; + $entincl{$key}{'content'} = ; + close IN; + $entincl{$key}{'length'} = ($entincl{$key}{'content'} =~ tr/\n/\n/); + print STDERR "read $filename (content of \&$key;, $entincl{$key}{'length'} lines long)\n" + if ($debug{'entities'}); + } + } + + # Change the entities including files in the document + my $dosubstitution = 1; + while ($dosubstitution) { + $dosubstitution = 0; + foreach my $key (keys %entincl) { + # The external entity can be referenced as &key; or &key + # In the second case, we must differentiate &key and &key2 + while ($origfile =~/^(.*?)&$key(;.*$|[^-_:.A-Za-z0-9].*$|$)/s) { + # Since we will include a new file, we + # must do a new round of substitutions. + $dosubstitution = 1; + my ($begin,$end)=($1,$2); + $end = "" unless (defined $end); + $end =~ s/^;//s; + + if ($begin =~ m/.*/s) { + # This entity is commented. Just remove it. + $origfile = $begin.$end; + next; + } + + # add the refs + my $len = $entincl{$key}{'length'}; # number added by the inclusion + my $pre = ($begin =~ tr/\n/\n/); # number of \n + my $post = ($end =~ tr/\n/\n/); + print "XX Add a ref. pre=$pre; len=$len; post=$post\n" + if $debug{'refs'}; + # Keep a reference of inclusion position in main file + my $main = $refs[$pre]; + + # Remove the references for the lines after the inclusion + # point. + my @endrefs = splice @refs, $pre+1; + + # Add the references of the added lines + my $i; + for ($i=0; $i<$len; $i++) { + $refs[$i+$pre] = "$main $entincl{$key}{'filename'}:".($i+1); + } + + if ($begin !~ m/\n[ \t]*$/s) { + if ($entincl{$key}{'content'} =~ m/^[ \t]*\n/s) { + # There is nothing in the first line of the + # included file, and something on the line before + # the inclusion The line reference will be more + # informative like this: + $refs[$pre] = $main; + } + } + if ($end !~ s/^[ \t]*\n//s) { + if ($entincl{$key}{'content'} =~ m/\n[ \t]*$/s) { + # There is something on the line after the + # inclusion, and there is an end of line at the + # end of the included file. We must add the line + # reference of the remainder on the line: + push @refs, $main; + } + } + # Append the references removed earlier (lines after the + # inclusion point). + push @refs, @endrefs; + + # Do the substitution + $origfile = "$begin".$entincl{$key}{'content'}."$end"; + print STDERR "substitute $key\n" if ($debug{'entities'}); + } + } + } + $origfile=~s/\G(.*?)&([A-Za-z_:][-_:.A-Za-z0-9]*|#[0-9]+|#x[0-9a-fA-F]+)\b/$1\{PO4A-amp\}$2/gs; + if (defined($xmlprolog) && length($xmlprolog)) { + $origfile=~s/\/>/\{PO4A-close\}>/gs; + } + + if ($debug{'refs'}) { + print "XX Resulting shifts\n"; + for (my $i=0; $i $refs[$i]\n"; + } + } + + my ($tmpfh,$tmpfile)=File::Temp::tempfile("po4a-XXXX", + SUFFIX => ".sgml", + DIR => "/tmp", + UNLINK => 0); + print $tmpfh $origfile; + close $tmpfh or die wrap_mod("po4a::sgml", dgettext("po4a", "Can't close tempfile: %s"), $!); + + my $cmd="nsgmls -l -E 0 -wno-valid < $tmpfile". + ($debug{'nsgmls'}?"":" 2>/dev/null")." |"; + print STDERR "CMD=$cmd\n" if ($debug{'generic'} or $debug{'nsgmls'}); + + open (IN,$cmd) || die wrap_mod("po4a::sgml", dgettext("po4a", "Can't run nsgmls: %s"), $!); + + # The kind of tags + my (%translate,%empty,%verbatim,%indent,%exist,%attribute,%qualify); + foreach (split(/ /, ($self->{SGML}->{k}{'translate'}||'') )) { + $translate{uc $_} = 1; + $indent{uc $_} = 1; + $exist{uc $_} = 1; + } + foreach (split(/ /, ($self->{SGML}->{k}{'empty'}||'') )) { + $empty{uc $_} = 1; + $exist{uc $_} = 1; + } + foreach (split(/ /, ($self->{SGML}->{k}{'verbatim'}||'') )) { + $translate{uc $_} = 1; + $verbatim{uc $_} = 1; + $exist{uc $_} = 1; + } + foreach (split(/ /, ($self->{SGML}->{k}{'indent'}||'') )) { + $translate{uc $_} = 1; + $indent{uc $_} = 1; + $exist{uc $_} = 1; + } + foreach (split(/ /, ($self->{SGML}->{k}{'ignore'}) || '')) { + $exist{uc $_} = 1; + } + foreach (split(/ /, ($self->{SGML}->{k}{'attributes'} || ''))) { + my ($attr, $tags); + if (m/(^.*>)(\w+)/) { + $attr=uc $2; + $tags=$1; + } else { + $attr=uc $_; + $tags=".*"; + } + if (exists $attribute{$attr}) { + $attribute{$attr}.="|$tags"; + } else { + $attribute{$attr} = $tags; + } + } + foreach (split(/ /, ($self->{SGML}->{k}{'qualify'}) || '')) { + $qualify{uc $_} = 1; + $attribute{uc $_} = '.*' unless exists $attribute{uc $_}; + } + + + # What to do before parsing + + # push the XML prolog if existing + $self->pushline($xmlprolog."\n") if (defined($xmlprolog) && length($xmlprolog)); + + # Put the prolog into the file, allowing for entity definition translation + # + # and push("translate("definition_of_my_entity") + if ($prolog =~ m/(.*?\[)(.*)(\]>)/s) { + warn "Pre=~~$1~~;Post=~~$3~~\n" if ($debug{'entities'}); + $self->pushline($1."\n") if (length($1)); + $prolog=$2; + my ($post) = $3; + while ($prolog =~ m/^(.*?)(.*)$/is) { #" ){ + $self->pushline($1) if length($1); + $self->pushline("translate($3,"","definition of entity \&$2;")."\">"); + warn "Seen text entity $2\n" if ($debug{'entities'}); + $prolog = $4; + } + $prolog .= $post; + $self->pushline($prolog."\n") if (length($prolog)); + } else { + warn "No entity declaration detected in ~~$prolog~~...\n" if ($debug{'entities'}); + $self->pushline($prolog) if length($prolog); + } + + # The parse object. + # Damn SGMLS. It makes me do crude things. + no strict "subs"; + my $parse= new SGMLS(IN); + use strict; + + # Some values for the parsing + my @open=(); # opened translation container tags + my $verb=0; # can we wrap or not + my $verb_last_ref; + my $seenfootnote=0; + my $indent=0; # indent level + my $lastchar = ''; # + my $buffer= ""; # what we will soon handle + + # Keep a reference to the last line indicated by nsgmls + my $line=0; + # Unfortunately, nsgmls do not mention all the line changes. We have + # to keep track of the number of lines seen in the "record ends". + my $adds=0; + # If the last line received contains only spaces, do not take it into + # account for the line reference of the paragraph. + my $empty_last_cdata=0; + # run the appropriate handler for each event + EVENT: while (my $event = $parse->next_event) { + # get the line reference to build po entries + if ($line != $parse->line) { + # nsgmls informs us of that the line changed. Reset $adds and + # $empty_last_cdata + $adds = 0; + $empty_last_cdata = 0; + $line = $parse->line; + } + my $ref=$refs[$parse->line-1 + $adds - $empty_last_cdata]; + # In verbatim mode, keep the current line reference. + if ($verb) { + $ref=$refs[$parse->line-1]; + } + my $type; + + if ($event->type eq 'start_element') { + die wrap_ref_mod($ref, "po4a::sgml", + dgettext("po4a", "Unknown tag %s"), + $event->data->name) + unless $exist{$event->data->name}; + + $lastchar = ">"; + + # Which tag did we see? + my $tag=''; + $tag .= '<'.lc($event->data->name()); + while (my ($attr, $val) = each %{$event->data->attributes()}) { + my $value = $val->value(); +# if ($val->type() eq 'IMPLIED') { +# $tag .= ' '.lc($attr).'="'.lc($attr).'"'; +# } els + if ($val->type() eq 'CDATA' || + $val->type() eq 'IMPLIED') { + if (defined $value && length($value)) { + my $lattr=lc $attr; + my $uattr=uc $attr; + if (exists $attribute{$uattr}) { + my $context=""; + foreach my $o (@open) { + next if (!defined $o or $o =~ m%^/; + $context.=$o; + } + $context=join("", $context, + "<", lc($event->data->name()), ">"); + if ($context =~ /^($attribute{$uattr})$/) { + if ($qualify{$uattr}) { + my $translated = $self->translate("$lattr=$value", $ref, "attribute $context$lattr"); + if ($translated =~ s/^$lattr=//) { + $value=$translated; + } else { + die wrap_mod("po4a::sgml", dgettext("po4a", "bad translation '%s' for '%s' in '%s'"), $translated, $context.$lattr, $ref); + } + } else { + $value = $self->translate($value, $ref, "attribute $context$lattr"); + } + } + } + if ($value =~ m/\"/) { + $value = "'".$value."'"; + } else { + $value = '"'.$value.'"'; + } + $tag .= " $lattr=$value"; + } + } elsif ($val->type() eq 'NOTATION') { + } else { + $tag .= ' '.lc($attr).'="'.lc($value).'"' + if (defined $value && length($value)); + } + } + $tag .= '>'; + + + # debug + print STDERR "Seen $tag, open level=".(scalar @open).", verb=$verb\n" + if ($debug{'tag'}); + + if ($event->data->name() eq 'FOOTNOTE') { + # we want to put the inside the in the same msgid + $seenfootnote = 1; + } + + if ($seenfootnote) { + $buffer .= $tag; + next EVENT; + } + if ($translate{$event->data->name()}) { + # Build the type + if (scalar @open > 0) { + $type=$open[$#open] . $tag; + } else { + $type=$tag; + } + + # do the job + if (@open > 0) { + $self->end_paragraph($buffer,$ref,$type,$verb,$indent, + @open); + } else { + $self->pushline($buffer) if $buffer; + } + $buffer=""; + push @open,$tag; + } elsif ($indent{$event->data->name()}) { + die wrap_ref_mod($ref, "po4a::sgml", dgettext("po4a", + "Closing tag for a translation container missing before %s"),$tag) + if (scalar @open); + } + + if ($verbatim{$event->data->name()}) { + $verb++; + # Keep a reference to the line that openned the verbatim + # section. This is needed to check if its data starts on + # the same line. + $verb_last_ref = $ref; + } + if ($verb) { + # Tag in a verbatim section. Check if it appeared at + # the same line than the previous data. If not, it + # means that an end of line must be added to the + # buffer. + if ($ref ne $verb_last_ref) { + # FIXME: Does it work if $verb > 1 + $buffer .= "\n"; + $verb_last_ref = $ref; + } + } + + if ($indent{$event->data->name()}) { + # push the indenting space only if not in verb before that tag + # push trailing "\n" only if not in verbose afterward + $self->pushline( ($verb>1?"": (" " x $indent)).$tag.($verb?"":"\n")); + $indent ++ unless $empty{$event->data->name()} ; + } else { + $tag =~ s///]]>/; + $buffer .= $tag; + } + } # end of type eq 'start_element' + + elsif ($event->type eq 'end_element') { + my $tag = ($empty{$event->data->name()} + ? + '' + : + 'data->name()).'>'); + + if ($verb) { + # Tag in a verbatim section. Check if it appeared at + # the same line than the previous data. If not, it + # means that an end of line must be added to the + # buffer. + if ($ref ne $verb_last_ref) { + # FIXME: Does it work if $verb > 1 + $buffer .= "\n"; + $verb_last_ref = $ref; + } + } + print STDERR "Seen $tag, level=".(scalar @open).", verb=$verb\n" + if ($debug{'tag'}); + + $lastchar = ">"; + + if ($event->data->name() eq 'FOOTNOTE') { + # we want to put the inside the in the same msgid + $seenfootnote = 0; + } + + if ($seenfootnote) { + $buffer .= $tag; + next EVENT; + } + if ($translate{$event->data->name()}) { + $type = $open[$#open] . $tag; + $self->end_paragraph($buffer,$ref,$type,$verb,$indent,@open); + $buffer = ""; + pop @open; + if (@open > 0) { + pop @open; + push @open,$tag; + } + } elsif ($indent{$event->data->name()}) { + die wrap_ref_mod($ref, "po4a::sgml", dgettext("po4a", + "Closing tag for a translation container missing before %s"), $tag) + if (scalar @open); + } + + unless ($event->data->name() =~ m/^(PO4ABEG|PO4AEND)$/si) { + if ($indent{$event->data->name()}) { + $indent -- ; + # add indenting space only when not in verbatim + # add the tailing \n only if out of verbatim after that tag + $self->pushline(($verb?"":(" " x $indent)).$tag.($verb>1?"":"\n")); + } else { + $buffer .= $tag; + } + $verb-- if $verbatim{$event->data->name()}; + } + } # end of type eq 'end_element' + + elsif ($event->type eq 'cdata') { + my $cdata = $event->data; + $empty_last_cdata=($cdata =~ m/^\s*$/); + $cdata =~ s/{PO4A-lt}//g; + $cdata =~ s/{PO4A-amp}/&/g; + $cdata =~ s/{PO4A-end}/\]\]>/g; + $cdata =~ s/{PO4A-beg-([^\}]+)}/PO4A-close\}>/\/>/sg; + $buffer =~ s/PO4A-close\}>//sg; # This should not be necessary + } + } # end of type eq 'cdata' + + elsif ($event->type eq 'sdata') { + my $sdata = $event->data; + $sdata =~ s/^\[//; + $sdata =~ s/\s*\]$//; + $lastchar = substr($sdata, -1, 1); + $buffer .= '&'.$sdata.';'; + } # end of type eq 'sdata' + + elsif ($event->type eq 're') { + # End of record, the line reference shall be incremented. + $adds +=1; + if ($verb) { + # Check if this line of data appear on the same line + # than the previous tag. If not, append an end of line + # to the buffer. + if ($ref ne $verb_last_ref) { + $buffer .= "\n"; + $verb_last_ref = $ref; + } + $buffer .= "\n"; + } elsif ($lastchar ne ' ') { + $buffer .= " "; + } + $lastchar = ' '; + } #end of type eq 're' + + elsif ($event->type eq 'conforming') { + + } + + elsif ($event->type eq 'pi') { + my $pi = $event->data; + $buffer .= ""; + } + + else { + die wrap_ref_mod($refs[$parse->line], "po4a::sgml", + dgettext("po4a","Unknown SGML event type: %s"), + $event->type); + } + } + + # What to do after parsing + $self->pushline($buffer); + close(IN); + warn wrap_mod("po4a::sgml", + dgettext("po4a","Warning: nsgmls produced some errors. ". + "This is usually caused by po4a, which modifies the input ". + "and restores it afterwards, causing the input of nsgmls ". + "to be invalid. This is usually safe, but you may wish ". + "to verify the generated document with nsgmls -wno-valid. ". + "Continuing...")) + if ($? != 0 and $self->verbose() > 0); + unlink ($tmpfile) unless ($debug{'refs'} or $debug{'nsgmls'}); +} + +sub end_paragraph { + my ($self, $para,$ref, $type,$verb,$indent)= + (shift,shift,shift,shift,shift,shift); + my (@open)=@_; + die "Internal error: no paragraph to end here!!" + unless scalar @open; + + return unless defined($para) && length($para); + + if (($para =~ m/^\s*$/s) and (not $verb)) { + # In non-verbatim environments, a paragraph with only spaces is + # like an empty paragraph + return; + } + + # unprotect &entities; + $para =~ s/{PO4A-amp}/&/g; + # remove the name"\|\|" nsgmls added as attributes + $para =~ s/ name=\"\\\|\\\|\"//g; + $para =~ s/ moreinfo=\"none\"//g; + + # Extract the leading and trailing spaces. They will be restored only + # in verbatim environments. + my ($leading_spaces, $trailing_spaces) = ("", ""); + if ($verb) { + # In the verbatim mode, we can ignore empty lines, but not the + # leading spaces or tabulations. Otherwise, the PO will look + # weird. + if ($para =~ m/^(\s*\n)(.*?)(\s*)$/s) { + $leading_spaces = $1; + $para = $2; + $trailing_spaces = $3; + } + } else { + if ($para =~ m/^(\s*)(.*?)(\s*)$/s) { + $leading_spaces = $1; + $para = $2; + $trailing_spaces = $3; + } + } + + $para = $self->translate($para,$ref,$type, + 'wrap' => ! $verb, + 'wrapcol' => (75 - $indent)); + + if ($verb) { + $para = $leading_spaces.$para.$trailing_spaces; + } else { + $para =~ s/^\s+//s; + my $toadd=" " x ($indent+1); + $para =~ s/^/$toadd/mg; + $para .= "\n"; + } + + $self->pushline( $para ); +} + +1; + +=head1 AUTHORS + +This module is an adapted version of sgmlspl (SGML postprocessor for the +SGMLS and NSGMLS parsers) which was: + + Copyright (c) 1995 by David Megginson + +The adaptation for po4a was done by: + + Denis Barbier + Martin Quinson (mquinson#debian.org) + +=head1 COPYRIGHT AND LICENSE + + Copyright (c) 1995 by David Megginson + Copyright 2002, 2003, 2004, 2005 by SPI, inc. + +This program is free software; you may redistribute it and/or modify it +under the terms of GPL (see the COPYING file). Index: branches/Bungo/package/bin/po4a/COPYING =================================================================== --- branches/Bungo/package/bin/po4a/COPYING (revision 0) +++ branches/Bungo/package/bin/po4a/COPYING (revision 2840) @@ -0,0 +1,341 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) 19yy + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., + 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) 19yy name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + , 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. Index: branches/Bungo/package/bin/po4a/README.translators =================================================================== --- branches/Bungo/package/bin/po4a/README.translators (revision 0) +++ branches/Bungo/package/bin/po4a/README.translators (revision 2840) @@ -0,0 +1,9 @@ +This package naturally propose to translate both its messages and its +documentation. The relevant po[t] files are in po/bin and po/pod +(respectively). + +In order to refresh them (sync them to the actual material to translate), +run from the top level: + + perl Build.PL + ./Build postats Index: branches/Bungo/package/bin/po4a/NEWS =================================================================== --- branches/Bungo/package/bin/po4a/NEWS (revision 0) +++ branches/Bungo/package/bin/po4a/NEWS (revision 2840) @@ -0,0 +1,571 @@ +po4a NEWS + +======================================================================= + +* Major changes in release 0.42 (UNRELEASED) + +Text: + * Add support for control files. + +======================================================================= + +* Major changes in release 0.41 (2010-12-01) + +po4a-updatepo: + * --previous flag was not passed to msgmerge + +po4a: + * Do no more generate backups for PO files. + * Fix the --srcdir option, master file was not found. + * Expand variables when reading @-prefixed addenda files. + +Text: + * Remove trailing newline in titles from PO files + +Packaging: + * Remove Makefile, all processing is performed by Build.PL + +======================================================================= + +* Major changes in release 0.40.2 (2010-11-30) + +documentation: + * Improve English style (following recommendations from man(7)) + +translations: + * Start of Danish translation + * Removal of Arabic translation, only one string was translated + * Updated Vietnamese translation + * Updated Slovenian translation + * Updated Basque translation + * Updated French translation + * Updated Japanese translation + * Updated Czech translation + * Updated German translation + * Updated Ukrainian translation + * Updated Esperanto translation + * Updated Estonian translation + * Updated Russian translation + +documentation translations: + * Updated Spanish translations + * Updated French translations + * Updated Japanese translations + * Updated Polish translations + +======================================================================= + +* Major changes in release 0.40.1 (2010-08-25) + +Po: + * Change header entry to be consistent with xgettext when creating + POT files: replace Content-Transfer-Encoding: ENCODING" by + "Content-Transfer-Encoding: 8bit\n" and add a Language field. + +documentation: + * Improve English style (proper casing, replace cvs by VCS, etc). + +translations: + * Updated French translations + * Updated Spanish translations + * Updated Japanese translations + * Updated Esperanto translations + * Updated Swedish translations + * Updated Russian translations + * Updated Estonian translations + * Updated Ukrainian translations + * Updated Czech translations + * Updated Portuguese translations + * Updated German translations + +Project resources: + * We migrated from CVS to SVN. All CVS history has been converted. + See http://svn.debian.org/viewsvn/po4a/ + The standard SVN layout is used (with top-level /trunk, /branches + and /tags directories), and the files to generate the website + (html/, po/html.cfg and po/www) are moved into a new /web + top-level directory. + +======================================================================= + +* Major changes in release 0.40 (2010-07-27) + +po4a-gettextize: + * When it fails, recode msgstr into current charset before + printing it. + +Text: + * Fix failures "Unknown option: copyright-holder" + +Man: + * Support font modifiers in the form \f(XX and \f[FONT-NAME] + +======================================================================= + +* Major changes in release 0.39 (2010-02-02) + +po4a + * New syntax for addenda, path may be preceded by modifiers. + +======================================================================= + +* Major changes in release 0.38 (2010-01-14) + +po4a + * Exit with an error if master file specified in a po4a configuration + file does not exist. + +po4a, po4a-updatepo: + * --previous is activated by default. + * Use --no-previous to support versions of gettext earlier than 0.16. + +Xml: + * Fix handling of multi-lines placeholder tags. + * New option addlang. + +Pod: + * Detect the encoding based on the input's =encoding line. + +======================================================================= + +* Major changes in release 0.37.1 (2009-11-22) + +po4a-build: + * Fix UTF8 handling and names of perl modules for section 3. + * Fix bug that caused translated manpages to be overwritten. + +======================================================================= + +* Major changes in release 0.37.0 (2009-11-20) + +po4a-build: + * New script for one-stop generation of untranslated and translated + documentation from DocBook XML and POD sources. + +translations: + * Updated Spanish translations. + +=============================================================================== +* Major changes in release 0.36.6 (2009-11-07) + +Text: + * Added options breaks and tabs. + +po4aman-display-po + * Fixed bashism. + +translations: + * Added Vietnamese translation + * Updated Czech translation + * Updated Spanish translation of the documentation + +=============================================================================== +* Major changes in release 0.36.5 (2009-09-10) + +Sgml: + * Fix detection of entities in the prolog. + +translations: + * Updated German translation + * Updated Portuguese translation + +=============================================================================== +* Major changes in release 0.36.4 (2009-08-30) + +po4a + * Added options --srcdir and --destdir. They permit to strip the base + directory for the input and output documents (in the config file and in + the references generated in the PO files). + +po4a-updatepo + * Added options --msgid-bugs-address, --copyright-holder, --package-name, + --package-version to control the generation of the PO header. + +translations: + * Updated Basque translation + * Updated Czech translation + * Updated Esperanto translation + * Improved Estonian translation. + * Updated French translation + * Updated Russian translation + * Updated Spanish translation. + * Updated Swedish translation. + +=============================================================================== +* Major changes in release 0.36.3 (2009-07-22) + +Bug fix release. +po4a-gettextize's new options were rejected by the modules. + +=============================================================================== +* Major changes in release 0.36.2 (2009-07-21) + +po4apod-display-po + * New script to display previews of POD translations. +po4a-gettextize + * Added options --msgid-bugs-address, --copyright-holder, --package-name, + --package-version to control the generation of the PO header. + +Text: + * Avoid the translation of some markups: title and horizontal rules. + * Improved Markdown support. +Xml: + * New option customtag for inline tags that should not be treated as a + tag (e.g. they do not need to be closed). +Wml: + * Fix handling of non ASCII input documents. + +=============================================================================== +* Major changes in release 0.36.1 (2009-04-05) + +TeX: + * Fix support for inline customization of separators. + * Fix support for double escapes, which should not introduce commands. +LaTeX: + * Added support for the tabularx environment. +Xml: + * Do not include commented out external entities (when includeexternal is + used). +Pod: + * Do not add an =encoding header if the encoding is empty. + +=============================================================================== +* Major changes in release 0.36 (2009-03-15) + +general: + * The "type:" comments in the PO files are now tagged as extracted + comments (starting with #.). This changes all the PO files generated by + po4a. Maintainers should just let these changes pass through. + Translator may want to remove the comment lines starting with '# type:' + as they will be duplicates with the '#. type:' comments. + +Xml: + * Placeholders are presented with a type and have a more valid XML + representation. This changes the PO files, but translations could be + updated automatically. + * Do not fail when the doctype does not match the expected one. Only + issue a warning. + * The tags and tagsonly options are deprecated. Use the + translated/untranslated and break/inline/placeholder options instead. + * Added foldattributes option. Useful to simplify strings and avoid typos + when attributes shall not be translated. +Man: + * Better handling of spaces in command arguments. +TeX: + * Included files are searched with kpsewhich instead of using TEXINPUTS + directly. + +=============================================================================== +* Major changes in release 0.35 (2009-02-10) + +po4a + * Added support for the [po_directory] command in the configuration file. + It permits to avoid listing the supported languages. Simply dropping a + new PO file should be sufficient. It should be preferred over the + po4a_langs and po4a_paths commands. +po4a-normalize: + * Added option -b, --blank to check which parts of a document are not + translated. +po4a-gettextize: + * Improved conflict handling: indicate the reference of the alternatives + when the same string has different translations. + +general: + * Added support for a nowrapi18n option in Locale::Po4a::Common in + order to use Locale::Po4a programatically. + +packaging: + * Manpages are generated in utf-8. This requires a recent Pod::Man with + support for the utf8 option. + +Docbook: + * Improved support for Docbook 5 and Docbook 4, based on the official + documentation of the Docbook tags. +Pod: + * Announce the encoding in the POD header. +Sgml: + * Add support for recursive inclusion. +Text: + * Added option asciidoc +Xhtml: + * Improvements for the tag. +xml: + * Fix the nodefault option. Derivative modules should use _default_tags + and _default_inline to define the default behavior of their + module-specific tags (instead of tags and inline). + * Speed improvements. + * Added support for placeholders. + +=============================================================================== +* Major changes in release 0.34 (2008-07-20) + +** New features +general: + * Do not end re-wrapped lines with a space. +halibut: + * Added support for the Halibut format. +sgml: + * Fix failures with big sgml documents ("broken pipe" received from + nsgmls -p). +text: + * Added support for the Markdown format, as used with Ikiwiki. + * Make sure files are not mixed together when gettextized at the same + time. +packaging: + * Fix build failures with old versions of Module::Build. + +=============================================================================== +* Major changes in release 0.33.3 (2008-04-02) + +po4a: + * Add options --package-name and --package-version (similar to xgettext's + options). + +translations: + * Actually ship Japanese translations. + +=============================================================================== +* Major changes in release 0.33.2 (2008-04-01) + +text: + * Added support for the fortunes format. + +translations: + * New Japanese translation. + * The manpages are generated in UTF-8. + +=============================================================================== +* Major changes in release 0.33.1 (2008-03-16) + +This is a bug fix release. + +man: + * Do not translate - to \- inside \*[...], \(.., \H'...', or \Z'...'. + +xml: + * Do not translate SSI comments by default. + +packaging: + * Fix the installation path of manpages and .mo files. + +=============================================================================== +* Major changes in release 0.33 (2008-03-03) + +Notes to packagers: Since 0.31, some tests in the sgml testsuite require +the docbook sgml DTD. If you run the testsuite at build time you must have +these DTD in your build dependencies or you must disable these tests +cases in the testsuite. + +** New features + general: Support for PO files with plural forms. + Po4a does not generate such PO files, but it can receive one in + input. In such case, po4a cannot choose between the (singular + and) plural forms. It will consider that msgstr[0] is the + translation of msgid and msgstr[1] is the translation of + msgid_plural, which may be wrong. Thus, a warning will be + issued if po4a is asked to translate the singular or plural + form of a message with plural forms. + + texinfo: Major changes. Tested on the elisp documentation. + + sgml: Fail if nsgmls cannot validate the input file. This can be turned + off with the "force" option. + + xml: New option "cpp" to support C preprocessor directives (this avoids + re-wrapping lines with preprocessor directives). + xml: New options "translated" and "untranslated" that should allow an + easier configuration of Xml (and derivative module) + xml: Avoid duplicated newlines at the end of lines of multi-lines + comments. + xml: Paragraphs which only consist in an inline tag are now extracted. + This makes a lot of string to available for translation in the PO + file. + docbook: Tested on the Docbook testsuite. Lots of tag added. + + po4a: Fix failures with perl 5.10. Thanks to Roderich Schupp + + + msguntypot: Fix handling of the comment of the PO header. + +=============================================================================== +* Major changes in release 0.32 (2007-08-15) + +** Security fix + Fix a symlink attack caused by the /tmp/gettextization.failed.po + temporary file. + (CVE-2007-4462) + +** Bug fixes + + sgml: The handling of newlines was broken in no-wrap sections in 0.31 + + xhtml: The module is now distributed as the other modules. + + texinfo: Fix infinite loop when a command parameter ends with \ + (as in @samp{\}) + +** New features + + xml: New option includeexternal to support external entities. + + texinfo: Added support for many commands and environment (see + changelog). + + xhtml: New option includessi, which adds supports for Server Side + Includes include element (). + + xml: New option ontagerror to control the behavior of the module in case + of error. This permits to support files that the module would + consider invalid otherwise. + +** Translations + + New translations (Bengali, Estonian, Croatian, Indonesian, Kannada, + Korean, Occitan, Uzbek, Simplified Chinese, Chinese from Hong Kong) and + many updates. + +=============================================================================== +* Major changes in release 0.31 (2007-05-07) + +** Dependencies + + po4a, + po4a-updatepo: New option --previous requires gettext 0.16. + +** Bug fixes + + general: Don't hang when running in background with redirected stdout. + Thanks to Jim Meyering. + +** New features + + general: po4a now uses timestamps to avoid re-generating a translation + if no changes are expected: if a translation is more recent + than its associated PO, master document, addenda or + configuration file, then there is no need to update it. For the + documents which do not pass the translation threshold, a + --stamp option was added to tell po4a to create files with a + .po4a-stamp extension (you can also create them manually). + This can save a lot of time when po4a knows that the + translation will not be generated because there were no changes + since the last po4a run. + + general: Beginning of a C extension. This provides speedup. Testers are + welcomed. + + po4a: New options: --msgid-bugs-address and --copyright-holder. + + po4a: Add new tag: [options] to set global options (i.e. for every + documents in the configuration file). + + po4a: Update translations based on modification times. Use timestamp to + avoid retrying generate uncompleted translations. + + man: New 'unknown_macros' option to specify the behavior of po4a with + unknown groff macros. + + man: Reset the configuration between two runs. This permits to define + different options for different files in a config file with the + opt:"..." parameters. + +=============================================================================== +* Major changes in release 0.30 (2007-01-05) + +** Bug fixes + + man: Recode the text which is skipped, like comments. + This kind of issue may also appear on other modules. + A better solution would be to always use UTF-8 internally. + + distribution: Some unit tests were missing. + Also, the dependencies have been simplified. + Locale::gettext, Text::WrapI18N, Term::ReadKey and SGMLS + are still recommended, but no more required. + The BibTex module is now distributed. + + sgml: The module should be more friendly with XML files. + + po4a: When po4a creates a new PO file, it now uses msginit. This may + change the header of created PO files. + + guide: Major update of the tag definitions. Thanks to Azamat H. + Hackimov. + + Po: use strftime instead of date because date is not available on + every platform. + + tests: rename the XML testsuite to SGML, since it was testing the Sgml + module, and add a new testsuite for the Xml module (it currently + tests the Guide module). + +** Translations + + Updated Russian translation. + +=============================================================================== +* Major changes in release 0.29 (2006-10-15) + +** New features + +** Bug fixes + + sgml: Remove the tags from the default categories when they are defined + by the user. This changes the behavior of the Sgml module's options. + + sgml: Better line references in the generated PO. + + sgml: Support for '/>' (empty XML tag closure) when an xml prologue is + found (). + + po4a: When po4a create new PO files, it now uses msginit instead of + copying the POT file. + +=============================================================================== +* Major changes in release 0.29 (2006-10-15) + +** New features + + New module for .INI files. Thanks to Costin Stroie. + + New module for BibTeX bibliographies. + + man: New mdoc option for stricter support of mdoc pages (NAME section + not translated). + + text: Support for underlined text. + + text: Support for bulleted paragraphs (can be deactivated with + -o nobullets). + +** Bug fixes + + man: Keep empty commented lines. + + man: Do not add spaces at the end of lines. + + man: Languages without non breaking space could not use question marks. + + sgml: Better support for verbatim sections, and consider + as verbatim. and