【问题标题】:How to create .avro file from a fixed width text file?如何从固定宽度的文本文件创建 .avro 文件?
【发布时间】:2012-08-16 10:15:25
【问题描述】:

我想从具有固定线宽的文本文件生成 .avro 文件。我是 java 和 .avro 的初学者,所以请帮我解决这个问题。

如何在 Java 中从固定宽度的文本文件创建 .avro 文件?

【问题讨论】:

  • fixed width file 是什么意思?
  • 固定宽度文件是一个简单的文本文件,具有列宽(字符)、填充、对齐(左/右)。我的意思是数据以有组织的方式排列。
  • 我认为固定宽度与文件无关,与编辑器有关。
  • 你看过这个帖子吗:stackoverflow.com/questions/5480308/…

标签: java fixed-width avro


【解决方案1】:

我使用 JRecord 包从固定宽度文件创建了一些 Avro 文件来读取固定宽度文件。在我的例子中,我使用 Cobol Copybook 来描述固定宽度文件。 JRecord 也可以使用 Xml 文件描述而不是 Cobol,两者的 java 代码实际上是相同的

示例: http://avroeditor.svn.sourceforge.net/viewvc/avroeditor/Source/AvroEditor/src/net/sf/RecordEditor/Avro/Test/IO/WriteSales.java?revision=4&view=markup

Avro 定义: http://avroeditor.svn.sourceforge.net/viewvc/avroeditor/IDL/Sales.genavro?revision=5&view=markup

JRecord: http://jrecord.sourceforge.net/JRecord04.html

或者,您可以使用其他平面文件阅读器之一来阅读固定宽度的文件,例如http://sourceforge.net/projects/flatpack/?source=directory

【讨论】:

    【解决方案2】:

    用 Javascript + jQuery 试试这个

    $( "#btn1" ).click(function() {
        $( "table" ).remove();
        var text1 = document.getElementById('text').value;
        var text1=text1.replace(/(\r\n|\n|\r)/gm,"");
        var sep = document.getElementById('sep').value;
        var aray = text1.split(sep);
        var formato = '<select class="form"><option value="string">string</option><option value="int">int</option><option value="date">date</option><option value="datetime">datetime</option><option value="float">float</option><option value="varchar(50)">varchar</option></select>';
        var univoco = '<input id="checkbox" type="checkbox" name="univoco" value="true" > <br>';
        var null1 = '<input id="null" class="null11" type="checkbox" name="univoco" value="null" checked> <br>';
        var html = '<table id="tbl" class="table table-sm  table-striped table-centered .thead-dark "><thead class="thead-dark"><tr class="table-primary"><th scope="col">ColumnName</th><th scope="col">Format</th>  <th scope="col">Null</th>   </tr>  </thead> <tbody>';
        $.each(aray, function(i, val) {
            html += '<tr><td id = "val">' + val + '</td><td id = "form">' + formato + '</td><td id = "nul">' + null1 + '</td></tr>';
        });
        html += '</tbody></table>'; 
        $("#table").append(html);
    });
    
    $( "#btn2").click(function() {
        $( "#avro" ).empty();
        var myRows = {myRows:[]};
        var $th = $('#tbl th');
        $('#tbl tbody tr').each(function(i, tr){
            var obj = {}, 
            $tds = $(tr).find('td');
            $td1 = $(tr).find('select.form').children("option:selected");
            $tds2 = $(tr).find('#checkbox');
            $tds3 = $(tr).find('#null');
            $th.each(function(){
                obj['name'] = $tds.eq(0).text();
                var type=$td1.eq(0).val();
                var nullv=$tds3.eq(0).is(':checked');
                if (type=='date') {
                    type={type:"int", logicalType:"date"};
                }else if (nullv == true) {
                    type = [type ,'null' ];
                }
                else{
                    type;
                }
                obj['type'] = type;
                // obj['univoco'] = $tds2.eq(0).is(':checked');
                // obj['null'] = $tds3.eq(0).is(':checked');
            });
            myRows.myRows.push(obj);        
        }); 
        console.log(JSON.stringify(myRows));
        var header = '{ <br> &nbsp &nbsp "type": "record"  ,  <br>   &nbsp &nbsp "namespace": "Mezzora" ,  <br>  &nbsp &nbsp "name": "ReportDSL" ,  <br>   &nbsp &nbsp  "fields": [ <br>';
        $('#avro').append(header);
        text ='&nbsp &nbsp &nbsp &nbsp &nbsp '+JSON.stringify(myRows.myRows[0])  +"<br>";
        $('#avro').append(text);
        var i;
        for (i = 1; i < myRows.myRows.length; i++) {
            text ='&nbsp &nbsp &nbsp &nbsp &nbsp  ,'+ JSON.stringify(myRows.myRows[i])  +"<br>";
            $('#avro').append(text);
        }
        var footer = '&nbsp &nbsp &nbsp &nbsp  ] <br> }';
        $('#avro').append(footer);  
    });
    
    $( "#btn3").click(function() {
        $( "#sql" ).empty();
        var myRows = {myRows:[]};
        var $th = $('#tbl th');
        $('#tbl tbody tr').each(function(i, tr){
            var obj = {}, 
            $tds = $(tr).find('td');
            $td1 = $(tr).find('select.form').children("option:selected");
            $tds2 = $(tr).find('#checkbox');
            $tds3 = $(tr).find('#null');
            $th.each(function(){
                obj['name'] = $tds.eq(0).text();
                var type=$td1.eq(0).val();
                var nullv=$tds3.eq(0).is(':checked');
                if (nullv == false) {
                    type= type +' not null';
                }else if (nullv == true) {
                    type = type +' null' ;
                }else{
                    if (type == 'string') {
                        type = 'varchar(50)';
                    }else{
                        type;
                    }
                }           
                obj['type'] = type;
                // obj['univoco'] = $tds2.eq(0).is(':checked');
                // obj['null'] = $tds3.eq(0).is(':checked');
            });
            myRows.myRows.push(obj);        
        }); 
        console.log(JSON.stringify(myRows));
        var header = 'CREATE TABLE [schema].[tblName] &nbsp ( <br>';
        $('#sql').append(header);
        var i;
        for (i = 0; i < myRows.myRows.length; i++) {
            text ='&nbsp &nbsp &nbsp &nbsp &nbsp '+ JSON.stringify(myRows.myRows[i].name).replace('\"',' ').replace('\"',' ') +JSON.stringify(myRows.myRows[i].type).replace('\"',' ').replace('\"',' ')   +", <br>";
            $('#sql').append(text);
        }
        var footer = ');';
        $('#sql').append(footer);   
    });
    .parent {
        display: grid;
        grid-template-columns: repeat(10, 1fr);
        grid-template-rows: repeat(12, 1fr);
        grid-column-gap: 20px;
        grid-row-gap: 0px;
        }
        
        .div1 { grid-area: 1 / 2 / 2 / 11; }
        .div2 { grid-area: 5 / 6 / 6 / 7; }
        .div3 { grid-area: 3 / 1 / 5 / 12; }
        .div4 { grid-area: 2 / 1 / 3 / 2; }
        .div5 { grid-area: 6 / 1 / 13 / 4; }
        .div6 { grid-area: 6 / 4 / 13 / 8; }
        .div7 { grid-area: 6 / 8 / 13 / 12; }
        
        .div6 , .div7{
    
            border: 1px solid black;
            margin-right: 10px;
        }
    
        #btn1{
            margin-bottom: 30px;
        }
    <!DOCTYPE html>
    <html lang="en">
    
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <meta http-equiv="X-UA-Compatible" content="ie=edge">
        <title></title>
      <script  src="https://code.jquery.com/jquery-3.4.1.js"  integrity="sha256-WpOohJOqMqqyKL9FccASB9O0KwACQJpFTUBLTYOVvVU="   crossorigin="anonymous"> </script>
      <link href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
    </head>
    <body>
        <h2>Insert the text and the separator and press the button: </h2>
        <div class="parent">
            <div class="div1"></div>
            <div class="div2 btn btn-alert"></div>
            <div class="div3">
          <textarea name="text" id="text" cols="120" rows="4" placeholder="ColumnName1,ColumnName2,ColumnName3,ColumnName4,............"></textarea>
          <input type="button"  value="Convert to Table" id="btn1" class="btn btn-primary"> 
        </div>
            <div class="div4">Separatetor<input type="text" name="sep" id="sep" value=","> </div>
            <div class="div5">
                <input type="button" value="Convert to Avro" class="btn btn-success" id="btn2">
                <input type="button" value="Convert to SQL Create" class="btn btn-info" id="btn3">
                <div id="table"></div>
            </div>
            <div class="div6 card">
                <div class="card-header font-weight-bolder">Avro Format</div>
                <div class="card-body" id='avro'>
                </div>
            </div>
            <div class="div7 card">
                <div class="card-header font-weight-bolder">SQL Create</div>
                <div class="card-body" id='sql'>
                </div>
            </div>
        </div>
    </body>
    </html>

    https://codepen.io/GentjanLikaj/pen/dyPXLXX

    【讨论】:

    • 问题需要 java 代码,而不是 javascript 代码。
    猜你喜欢
    • 2014-02-22
    • 1970-01-01
    • 2011-08-12
    • 2013-01-01
    • 1970-01-01
    • 2021-10-24
    • 2016-01-18
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多