我想使用 WordNet 查找单词相似度

Question

我正在做关于“网络视频分类”的最后一年项目，其中一部分是找到特定单词的相似（同义词）词，我想从中删除相似的词。

我知道 Java 语言，所以我选择了“Java 的单词相似度”ws4j

为此，我只使用了WS4J1.0.1 jar 文件，甚至我还没有下载任何额外的文件，如 WordNet 词法数据库或 sqlite 数据库来存储它。因为在那个网站上，他们提到所有都包含在这个 jar 文件中。

当我执行Demo程序SimilarityCalculationDemo.java时，出现如下错误：

    java.sql.BatchUpdateException: batch entry 0: [SQLITE_CORRUPT]  The database disk image is malformed (database disk image is malformed)
    at org.sqlite.Stmt.executeBatch(Stmt.java:226)
    at org.sqlite.Stmt.executeBatch(Stmt.java:226)
    at edu.cmu.lti.jawjaw.db.SQL.createIndexIfNotExists(SQL.java:118)
    at edu.cmu.lti.jawjaw.db.SQL.createSQLConnection(SQL.java:98)
    at edu.cmu.lti.jawjaw.db.SQL.<init>(SQL.java:55)
    at edu.cmu.lti.jawjaw.db.SQL.<clinit>(SQL.java:45)
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:124)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
                             atedu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
        java.sql.SQLException: [SQLITE_CORRUPT]  The database disk image is malformed (database disk image is malformed)
    at org.sqlite.DB.newSQLException(DB.java:383)
    at org.sqlite.DB.newSQLException(DB.java:387)
    at org.sqlite.DB.throwex(DB.java:374)
    at org.sqlite.NativeDB.prepare(Native Method)
    at org.sqlite.DB.prepare(DB.java:123)
    at org.sqlite.Stmt.execute(Stmt.java:113)
    at edu.cmu.lti.jawjaw.db.SQL.setPragmaCacheSize(SQL.java:137)
    at edu.cmu.lti.jawjaw.db.SQL.createSQLConnection(SQL.java:99)
    at edu.cmu.lti.jawjaw.db.SQL.<init>(SQL.java:55)
    at edu.cmu.lti.jawjaw.db.SQL.<clinit>(SQL.java:45)
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:124)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
    at edu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
java.sql.SQLException: [SQLITE_CORRUPT]  The database disk image is malformed (database disk image is malformed)
    at org.sqlite.DB.newSQLException(DB.java:383)
    at org.sqlite.DB.newSQLException(DB.java:387)
    at org.sqlite.DB.throwex(DB.java:374)
    at org.sqlite.NativeDB.prepare(Native Method)
    at org.sqlite.DB.prepare(DB.java:123)
    at org.sqlite.PrepStmt.<init>(PrepStmt.java:42)
    at org.sqlite.Conn.prepareStatement(Conn.java:404)
    at org.sqlite.Conn.prepareStatement(Conn.java:399)
    at org.sqlite.Conn.prepareStatement(Conn.java:383)
    at edu.cmu.lti.jawjaw.db.SQL.prepareStatements(SQL.java:151)
    at edu.cmu.lti.jawjaw.db.SQL.<init>(SQL.java:56)
    at edu.cmu.lti.jawjaw.db.SQL.<clinit>(SQL.java:45)
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:124)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
    at edu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
Exception in thread "main" java.lang.NullPointerException
    at edu.cmu.lti.jawjaw.db.WordDAO.findWordsByLemmaAndPos(WordDAO.java:125)
    at edu.cmu.lti.jawjaw.util.WordNetUtil.wordToSynsets(WordNetUtil.java:38)
    at edu.cmu.lti.lexical_db.NictWordNet.getAllConcepts(NictWordNet.java:38)
    at edu.cmu.lti.ws4j.util.WordSimilarityCalculator.calcRelatednessOfWords(WordSimilarityCalculator.java:79)
    at edu.cmu.lti.ws4j.RelatednessCalculator.calcRelatednessOfWords(RelatednessCalculator.java:61)
    at web_cat.SimilarityCalculationDemo.run(SimilarityCalculationDemo.java:37)
    at web_cat.SimilarityCalculationDemo.main(SimilarityCalculationDemo.java:43)
Java Result: 1

我正在使用带有 JDK 6 的 Netbeans IDE 7.4。

任何人都可以帮助我，如何解决这个问题，因为互联网上关于 ws4j 的文档较少。

Answer 1

好吧，我无法重现您的错误。对我来说，它开箱即用，使用 eclipse，所以我会尽力帮助你准确重现我所做的

1. 从https://ws4j.googlecode.com/files/ws4j-1.0.1.jar 下载 ws4j-1.0.1.jar 并确保下载后的大小为 41,362,723 字节（至少，这是 eclipse 在我的 linux 机器中告诉我的）

2. 使用 java 7

3. 创建一个简单的 Eclipse 项目并将 jar 放在那里。然后将jar添加到构建路径 （右键->构建路径->添加）

4. 创建适当的包和类以容纳演示类

5. 只要运行演示，你就会得到类似的东西

   edu.cmu.lti.ws4j.impl.HirstStOnge   0.0
   edu.cmu.lti.ws4j.impl.LeacockChodorow   1.3862943611198906
   edu.cmu.lti.ws4j.impl.Lesk  0.0
   edu.cmu.lti.ws4j.impl.WuPalmer  0.4
   edu.cmu.lti.ws4j.impl.Resnik    2.5031573470157453
   edu.cmu.lti.ws4j.impl.JiangConrath  0.11150424023847051
   edu.cmu.lti.ws4j.impl.Lin   0.3582442863008455
   edu.cmu.lti.ws4j.impl.Path  0.14285714285714285
   Done in 1951 msec.