Skip to content

Fix Deep Learning header classification inconsistency #1211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 21 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
d95a1ac
first attempt to fix tables on the fly
lfoppiano Dec 4, 2024
b012665
revise table validation, apply check to figures, move code outside th…
lfoppiano Dec 6, 2024
d2ca7c9
fix figure validation
lfoppiano Dec 6, 2024
575c2fe
move test related to LabelUtils outside
lfoppiano Dec 6, 2024
7780804
fix scope
lfoppiano Dec 6, 2024
e0e217d
remove unnecessary layout tokens list, renaming stuff
lfoppiano Dec 6, 2024
facc35e
fix index mismatch
lfoppiano Dec 6, 2024
17ad425
fix wrong label adjustment
lfoppiano Dec 6, 2024
c830231
fix wrong upper indexes, cleanup
lfoppiano Dec 7, 2024
9c77194
Merge branch 'refs/heads/master' into bugfix/fix-dropped-misclassifie…
lfoppiano Dec 16, 2024
2f0661b
cosmetics
lfoppiano Dec 16, 2024
2f04ccd
add tests for labelling adjustment
lfoppiano Dec 16, 2024
f5eb758
tests and refactoring in smaller pieces
lfoppiano Dec 17, 2024
81a1691
fix typo
lfoppiano Dec 17, 2024
3778a6e
add test for table token consolidation
lfoppiano Dec 17, 2024
21f85c9
rename methods for better clarity, move utility methods in Kotlin
lfoppiano Dec 18, 2024
1fa792c
fix header sequence with delft to revert any labels that do not have …
lfoppiano Dec 18, 2024
cbbe460
renaming
lfoppiano Dec 19, 2024
f036e0a
allow a loose approach to gather table/figure starting token when the…
lfoppiano Dec 19, 2024
09c824d
improve the way the candidate identification for bad figures/tables i…
lfoppiano Dec 19, 2024
77cf7a2
Merge branch 'bugfix/fix-dropped-misclassified-text' into bugfix/fix-…
lfoppiano Dec 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
first attempt to fix tables on the fly
  • Loading branch information
lfoppiano committed Dec 4, 2024
commit d95a1ac53cde69ed00b46a68776a39008fea5797
11 changes: 11 additions & 0 deletions grobid-core/src/main/java/org/grobid/core/data/Table.java
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@
public class Table extends Figure {
private List<LayoutToken> contentTokens = new ArrayList<>();
private List<LayoutToken> fullDescriptionTokens = new ArrayList<>();

// Contains the raw layoutTokens from the fulltext model
private List<LayoutToken> rawLayoutTokens = new ArrayList<>();
private boolean goodTable = true;

private StringBuilder note = null;
Expand Down Expand Up @@ -423,4 +426,12 @@ public boolean isGoodTable() {
public String getTeiId() {
return "tab_" + this.id;
}

public List<LayoutToken> getRawLayoutTokens() {
return rawLayoutTokens;
}

public void setRawLayoutTokens(List<LayoutToken> rawLayoutTokens) {
this.rawLayoutTokens = rawLayoutTokens;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -874,6 +874,7 @@ public static List<GraphicObject> getConnectedGraphics(Block block, Document doc
public void postProcessTables() {
for (Table table : tables) {
if (!table.firstCheck()) {
table.setGoodTable(false);
continue;
}

Expand Down Expand Up @@ -919,7 +920,7 @@ public void postProcessTables() {
table.getContentTokens().clear();
table.getContentTokens().addAll(contentResult);

table.secondCheck();
table.setGoodTable(table.secondCheck());
}
}

Expand Down
Loading
Loading