--======== Review Reports ========--
The review report from reviewer #1:
*1: Is the paper relevant to ICDM?
[_] No
[X] Yes
*2: How innovative is the paper?
[_] 6 (Very innovative)
[_] 3 (Innovative)
[_] -2 (Marginally)
[X] -4 (Not very much)
[_] -6 (Not at all)
*3: How would you rate the technical quality of the paper?
[_] 6 (Very high)
[_] 3 (High)
[_] -2 (Marginal)
[X] -4 (Low)
[_] -6 (Very low)
*4: How is the presentation?
[_] 6 (Excellent)
[_] 3 (Good)
[X] -2 (Marginal)
[_] -4 (Below average)
[_] -6 (Poor)
*5: Is the paper of interest to ICDM users and practitioners?
[X] 3 (Yes)
[_] 2 (May be)
[_] 1 (No)
[_] 0 (Not applicable)
*6: What is your confidence in your review of this paper?
[X] 2 (High)
[_] 1 (Medium)
[_] 0 (Low)
*7: Overall recommendation
[_] 6: must accept (in top 25% of ICDM accepted papers)
[_] 3: should accept (in top 80% of ICDM accepted papers)
[_] -2: marginal (in bottom 20% of ICDM accepted papers)
[X] -4: should reject (below acceptance bar)
[_] -6: must reject (unacceptable: too weak, incomplete, or wrong)
*8: Summary of the paper's main contribution and impact
The authors consider large Twitter data to analyze relationships and sentiment between several countries and China. They consider tweets mentioning one of 7 countries and several big Chinese companies, and consider how positive/negative the sentiment is in each and towards which company.
*9: Justification of your recommendation
Please see answer 18 below.
*10: Three strong points of this paper (please number each point)
- Interesting and timely analysis.
- Data to be provided, and the analysis is in general quite reproducible.
- Some interesting insights provided.
*11: Three weak points of this paper (please number each point)
- The title and the text don't fully match.
- The analysis is quite shallow.
- The paper reads like a draft of a larger study and is not ready for conference publication.
*12: Is this submission among the best 10% of submissions that you reviewed for ICDM'21?
[X] No
[_] Yes
*13: Are the datasets used in the study correctly identified and referenced?
[_] 3 Yes
[X] 2 Partial
[_] 1 No
[_] 0 Not applicable
*14: Are the competing methods used in the study correctly identified and referenced?
[_] 3 Yes
[_] 2 Partial
[_] 1 No
[X] 0 Not applicable
*15: Is the source code for the proposed method made publicly available?
[_] 3 Yes
[_] 2 Partial
[_] 1 No
[X] 0 Not applicable
*16: Is the experimental design detailed enough to allow for reproducibility? (You can also include comments on reproducibility in the body of your review.)
[X] 3 Yes
[_] 2 Partial
[_] 1 No
[_] 0 Not applicable
*17: If the paper is accepted, which format would you suggest?
[_] Regular Paper
[X] Short Paper
*18: Detailed comments for the authors
The authors provide quite an interesting study of political communication. However, the paper feels unfinished, and the study and analysis is shallow at best. Please find detailed comments below:
- The authors frame the intro in the context of US-China relationship, yet the study does not limit itself to those 2 countries.
- In addition, they frame it as an analysis of disputes and political communication, yet the tweets are not necessarily of a political topic, and the disputes are not really analyzed, just the sentiment.
- English is strange at times and should be ironed out.
- The pictures are very small, not visible when printed.
- The experiments are not explained, e.g., how was word2vec even trained is not discussed at all.
- How were 2D plots made, that is also not explained.
- There are a lot more pages left to write, and the authors should extend both the explanations and the analysis quite a lot.
- The empirical analysis is limited to several relatively straightforward plots/graphs, and deeper insights and analysis is required.
========================================================
The review report from reviewer #2:
*1: Is the paper relevant to ICDM?
[_] No
[X] Yes
*2: How innovative is the paper?
[_] 6 (Very innovative)
[_] 3 (Innovative)
[X] -2 (Marginally)
[_] -4 (Not very much)
[_] -6 (Not at all)
*3: How would you rate the technical quality of the paper?
[_] 6 (Very high)
[_] 3 (High)
[_] -2 (Marginal)
[_] -4 (Low)
[X] -6 (Very low)
*4: How is the presentation?
[_] 6 (Excellent)
[_] 3 (Good)
[_] -2 (Marginal)
[_] -4 (Below average)
[X] -6 (Poor)
*5: Is the paper of interest to ICDM users and practitioners?
[_] 3 (Yes)
[X] 2 (May be)
[_] 1 (No)
[_] 0 (Not applicable)
*6: What is your confidence in your review of this paper?
[X] 2 (High)
[_] 1 (Medium)
[_] 0 (Low)
*7: Overall recommendation
[_] 6: must accept (in top 25% of ICDM accepted papers)
[_] 3: should accept (in top 80% of ICDM accepted papers)
[_] -2: marginal (in bottom 20% of ICDM accepted papers)
[_] -4: should reject (below acceptance bar)
[X] -6: must reject (unacceptable: too weak, incomplete, or wrong)
*8: Summary of the paper's main contribution and impact
This study analyzed the quantitative characteristics, linguistic preferences and sentiment of the tweets that simultaneously contains names or synonyms of at least one of the seven English-speaking countries and the names or synonyms of at least one of the three Chinese tech giants. The idea is interesting.
*9: Justification of your recommendation
Although the idea behind this study is interesting, I am not convinced that the authors did what they intended to do based on their title and the main text. They intended to study the US-China disputes on Chinese tech giants, however, they collected and kept the tweets as long as they contained names of one of the seven English-speaking countries and one of the three Chinese tech giants simultaneously. Talking about one English-speaking country and one of the Chinese tech giants does not necessarily mean that the tweet is about the US-China dispute. Based on this data collection, I am not convinced of any of the findings.
*10: Three strong points of this paper (please number each point)
1. The high-level idea behind this study is interesting.
2. The "Existing Work" section is well-written.
3. The topic is relevant to ICDM.
*11: Three weak points of this paper (please number each point)
1. The collected data may not be related to US-China dispute on Chinese tech giants, which may lead to wrong conclusions.
2. The presentation of the figures is poor. It is hard to read Figure 1. Labels of x- and y- axis are missing in Figure 2. Although there is still plenty of room left (10 pages maximum, authors only used 6 pages), the authors did not generate a good layout of the figures.
3. Some statements need citations to support. For example, in II.B., "According to a Pew Research Center report, ..." This statement needs citations.
*12: Is this submission among the best 10% of submissions that you reviewed for ICDM'21?
[X] No
[_] Yes
*13: Are the datasets used in the study correctly identified and referenced?
[_] 3 Yes
[_] 2 Partial
[_] 1 No
[X] 0 Not applicable
*14: Are the competing methods used in the study correctly identified and referenced?
[_] 3 Yes
[_] 2 Partial
[_] 1 No
[X] 0 Not applicable
*15: Is the source code for the proposed method made publicly available?
[_] 3 Yes
[_] 2 Partial
[X] 1 No
[_] 0 Not applicable
*16: Is the experimental design detailed enough to allow for reproducibility? (You can also include comments on reproducibility in the body of your review.)
[_] 3 Yes
[_] 2 Partial
[_] 1 No
[X] 0 Not applicable
*17: If the paper is accepted, which format would you suggest?
[_] Regular Paper
[X] Short Paper
*18: Detailed comments for the authors
My main concern is about the data collection, which I have provided details in question 9 ("Justification of your recommendation"). As I discussed, mentioning an English-speaking country and a Chinese tech giant does not necessarily mean the tweet is about the US-China disputes on Chinese tech giants. I am not convinced that the conclusions drawn from these data are related to this paper's topic.
In addition to three weak points that I have listed before, there are a few minor concerns:
(1) Typo: "This study uses the data form March 4, ..." -- > "from"
(2) Inconsistent usage of acronyms: in some places the authors used "UK", while in some places the authors used "U.K."
(3) Typo: "twitter" -- > "Twitter"
(4) Normally, when we use the word "significantly", it requires statistical tests. However, the authors claimed "significantly" without any statistical evidence.
(5) The authors used an unsupervised sentiment analysis algorithm developed by another study. It is strongly recommended that the authors validate the accuracy of this algorithm using their own data.
========================================================
The review report from reviewer #3:
*1: Is the paper relevant to ICDM?
[_] No
[X] Yes
*2: How innovative is the paper?
[_] 6 (Very innovative)
[_] 3 (Innovative)
[_] -2 (Marginally)
[X] -4 (Not very much)
[_] -6 (Not at all)
*3: How would you rate the technical quality of the paper?
[_] 6 (Very high)
[_] 3 (High)
[_] -2 (Marginal)
[X] -4 (Low)
[_] -6 (Very low)
*4: How is the presentation?
[_] 6 (Excellent)
[_] 3 (Good)
[X] -2 (Marginal)
[_] -4 (Below average)
[_] -6 (Poor)
*5: Is the paper of interest to ICDM users and practitioners?
[_] 3 (Yes)
[X] 2 (May be)
[_] 1 (No)
[_] 0 (Not applicable)
*6: What is your confidence in your review of this paper?
[X] 2 (High)
[_] 1 (Medium)
[_] 0 (Low)
*7: Overall recommendation
[_] 6: must accept (in top 25% of ICDM accepted papers)
[_] 3: should accept (in top 80% of ICDM accepted papers)
[_] -2: marginal (in bottom 20% of ICDM accepted papers)
[X] -4: should reject (below acceptance bar)
[_] -6: must reject (unacceptable: too weak, incomplete, or wrong)
*8: Summary of the paper's main contribution and impact
This paper introduces the prospect of computational political communication, which means analyzing political issues by automatic collecting and reading the discourse on social media. A quantitative experiment is conducted by utilizing a Twitter dataset.
*9: Justification of your recommendation
There is very little or even no technical contribution in this work. Besides, the methodology in this paper is also questionable. Last but not least, the experiments are result aggregations that are not very insightful in practice.
*10: Three strong points of this paper (please number each point)
S1: Detailed literature review
S2: The topic has potential practical values
*11: Three weak points of this paper (please number each point)
W1: Lack of technical contribution
W2: Questionable methodology
W3: No insightful experiment results
*12: Is this submission among the best 10% of submissions that you reviewed for ICDM'21?
[X] No
[_] Yes
*13: Are the datasets used in the study correctly identified and referenced?
[X] 3 Yes
[_] 2 Partial
[_] 1 No
[_] 0 Not applicable
*14: Are the competing methods used in the study correctly identified and referenced?
[_] 3 Yes
[_] 2 Partial
[_] 1 No
[X] 0 Not applicable
*15: Is the source code for the proposed method made publicly available?
[X] 3 Yes
[_] 2 Partial
[_] 1 No
[_] 0 Not applicable
*16: Is the experimental design detailed enough to allow for reproducibility? (You can also include comments on reproducibility in the body of your review.)
[X] 3 Yes
[_] 2 Partial
[_] 1 No
[_] 0 Not applicable
*17: If the paper is accepted, which format would you suggest?
[_] Regular Paper
[X] Short Paper
*18: Detailed comments for the authors
First it should improve some fundamentals concerning the methodologies:
1. Given the title “U.S.-China Disputes”, what is the reason for including other 6 English speaking countries, especially Pakistan which isn’t conventionally considered as an ally of the other countries here;
2. The English-speaking population in India and Pakistan only makes
up about 10% of the whole population. Therefore, it’s doubtful if the tweets can reflect the public opinions in these two countries.
3. The names of companies include Chinese names like “zijietiaodong”, which almost limits the observation to Chinese users only.
4. The experiments don’t reveal whose opinion towards whom — simultaneously including a country name and a company name neither reveal the user ’s identity nor attitude towards which objective.
5. Detection for spammer and fake users is necessary for platforms like Twitter.
========================================================
Meta Review:
The reviewers agree that this is a relevant and timely topic, and they see merit in the proposed approach. However, they also point out that the paper in its current version has several weaknesses that will need thorough revision. In particular, they point out
That novelty is limited
That the proposal is an interesting application, but that the paper has too little technical novelty to fit ICDM. A venue more focused on the application area may be a better place for this work.
The proposal has conceptual and procedural challenges (motivation for selection of countries, explanation of some variables values, dubious operationalization of the concept of interest that would require independent validation of a large-enough dataset, need for data cleaning, lack of detail in the mined opinion) that, taken together, lead to too little evidence for the drawn conclusions.
Some found the argumentation difficult to follow and recommend a thorough revision of the writing.
The reviewers point out that the manuscript is in need of proofreading.
In sum, the paper does not appear ready yet for publication in ICDM.
We hope that the reviewers’ comments will be helpful for you to prepare an improved version of your paper.
Further work is encouraged.