adaptive IE tool














Experimental Results


Fabio Ciravegna, Department of Computer Science, University of Sheffield
[F.Ciravegna@dcs.shef.ac.uk]

   

Experimental Results


Amilcare has been tested on a number of corpora. Here are the results on two well known corpora, i.e. the CMU seminar announcement proposed by Freitag and the Job Postings proposed by M. E. Califf. For results on teh Pascal challenge corpus, see here.

Parameters used for Seminar Announcements and Job Postings

Parameter

Value

Pattern Length

4

Consider EOL

YES

Use Ontology

NO

MinMatches for Tagging Rules

1

Error Threshold

0.95

Results for Seminar Announcements

(Cross-validation on 5 random 50/50 splits of the data)

Slot

Partial Matches

Exact Matches

P

R

F 1

P

R

F 1

Speaker

90.38

87.95

89.11

86.64

84.33

85.44

Location

91.78

75.52

82.83

85.51

70.37

77.18

Stime

96.07

93.52

94.77

94.59

92.08

93.32

Etime

98.13

96.92

97.52

97.11

95.91

96.50

Macro-average

94.09

88.48

91.06

90.96

85.67

88.11

Micro-average

93.83

88.38

90.90

90.70

85.56

87.94

 

Results for Job Postings

(Cross-validation on 5 random 50/50 splits of the data)

Slot

Partial Matches

Exact Matches

P

R

F 1

P

R

F 1

id

99.20

99.47

99.33

98.80

99.07

98.93

title

73.08

49.48

58.89

56.75

38.38

45.71

company

85.09

80.85

82.73

80.09

76.07

77.86

salary

88.10

68.92

77.21

80.13

62.71

70.24

recruiter

88.02

77.53

82.39

83.52

73.50

78.15

state

95.30

99.11

97.16

94.78

98.57

96.63

city

95.74

98.56

97.13

93.79

96.55

95.15

country

98.22

98.72

98.47

98.16

98.66

98.41

language

84.01

82.03

82.98

76.65

74.83

75.70

platform

77.90

74.23

75.93

69.18

65.89

67.41

application

82.36

83.23

82.74

76.29

77.07

76.64

area

69.29

54.21

60.78

59.23

46.33

51.95

req_yrs_exp

85.10

86.90

85.84

82.40

84.15

83.12

des_yrs_exp

89.76

93.55

91.51

87.28

90.96

88.97

req_degree

93.83

87.92

90.76

92.88

87.02

89.84

des_degree

89.39

51.17

64.55

85.64

48.67

61.55

post_date

97.98

100.00

98.97

97.98

100.00

98.97

Macro-average

87.79

81.52

83.96

83.15

77.55

79.72

Micro-average

84.64

79.55

81.67

78.65

74.37

76.15

 

 

 
<< Back Next >>
 

Last updated: November 24, 2002