{ "id": 5454, "title": "Power measurements?", "url": "/competitions/belkin-energy-disaggregation-competition/discussion/5454", "author_name": "", "post_date": "2013-08-19T21:09:12.733Z", "votes": null, "comment_count": 7, "views": 2214, "content": "

Hi,

\n

I have a question about the source of these measurements.

\n

In the low frequency data provided, we receive complex values for the voltage and current across each of the mains. However, if the data is taken from a device like that outlined in [4], it doesn't seem possible to measure the current through the mains.

\n

I started wondering if these measurements for current actually reflect the current through a device similar to the ElectriSense sensor, rather than taken in by the house itself. More details would be helpful.

\n

--

\n

Also, could we receive a little more detail as to what information is in the high frequency data? It is just a large time-indexed vector as is. I understand that it is just the magnitude of the spectrogram of some signal; however, it is unclear whether or not it contains information on the voltage or the power, or information on phases/mains 1, 2, or both.

\n

Thanks in advance.

", "messages": [ { "id": "28985", "postDate": "08/19/2013 21:09:12", "content": "

Hi,

\n

I have a question about the source of these measurements.

\n

In the low frequency data provided, we receive complex values for the voltage and current across each of the mains. However, if the data is taken from a device like that outlined in [4], it doesn't seem possible to measure the current through the mains.

\n

I started wondering if these measurements for current actually reflect the current through a device similar to the ElectriSense sensor, rather than taken in by the house itself. More details would be helpful.

\n

--

\n

Also, could we receive a little more detail as to what information is in the high frequency data? It is just a large time-indexed vector as is. I understand that it is just the magnitude of the spectrogram of some signal; however, it is unclear whether or not it contains information on the voltage or the power, or information on phases/mains 1, 2, or both.

\n

Thanks in advance.

", "rawMarkdown": "", "votes": null }, { "id": "29097", "postDate": "08/20/2013 19:50:56", "content": "

As I understand it, the high frequency data is a matrix with each line a vector of 4096 values. One vector for each time point. Each vector is calculated by a Fourier Transform of the original noise signals measured by the hardware. Thus, each vector in the HF data is a spectrum of high frequency noise (just like a rainbow is a spectrum of light). Each value in the vector is the magnitude or intensity of the noise at that frequency. The papers suggest that modern power-supplies have a noise signature: a characteristic pattern of high frequency noise intensities, and that these patterns can be used to distinguish between household devices. 

", "rawMarkdown": "", "votes": null }, { "id": "29114", "postDate": "08/21/2013 04:09:28", "content": "

Yes, I agree completely. The data comes from the signal in the frequency domain. My second question is asking what the signal actually is. Is it a voltage signal or a power signal? Is it the signal in phase 1, phase 2, both phases measured independently and concatenated into one vector, or the sum across both phases? (Note that the low frequency data is divided into phase 1 and phase 2 signals.)

", "rawMarkdown": "", "votes": null }, { "id": "29117", "postDate": "08/21/2013 05:14:03", "content": "

I don't think that the low frequency data is actually 'divided into phase1 and phase2 signals'. If you look at the Buffer data, which is what is actually loaded from the file, it consists of current (I) and voltage (V) on two lines, L1 and L2. Both measurements are at multiple harmonics of the 60Hz line frequency. Each datapoint is a complex number. The HF data is separate. My understanding is that it corresponds to high frequency noise measurements or EMI. The sampling rate needed for these measurements is much higher than the TimeTicks in either L1 or L2. (1MHz, or 1e6/second). The FFT makes the noise data considerably smaller than the raw form. 

\n

Have a good look at the reference from UbiComp, 2010 (ElectriSense: Single-Point Sensing Using EMI for Electrical Event Detection and Classification in the Home by Sidhant Gupta, Matthew S. Reynolds, Shwetak N. Patel). It has a circuit diagram and a full description of the data and analyses tried earlier. 

\n


", "rawMarkdown": "", "votes": null }, { "id": "29120", "postDate": "08/21/2013 05:40:07", "content": "

What I meant by 'phases', which seems to be the term this data set uses, is what you're calling 'lines'. (The term appears on this page: https://www.kaggle.com/c/belkin-energy-disaggregation-competition/data) I believe they're usually referred to as 'mains', but phase makes sense since they, ideally, would be perfectly out of phase. So while the low frequency data does have information for lines 1 and 2, the high frequency has no such division explicitly present.

\n

Your comments bring me back to my first question, which is that the ElectriSense sensor could not possibly measure the current through the mains itself.

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "29125", "postDate": "08/21/2013 06:39:07", "content": "

Oh. I get it. No, I don't know how they're measuring the current, but I suspect that is part of the Belkin proprietary hardware. 

\n

For the HF measurement:(wikipedia) Electromagnetic_interference. This is primarily about radiated or RF interference, but similar effects happen on power lines.

\n

You might also try searching for 'ground loop' or 'ground loop interference'.  With consumer devices, we don't usually run into trouble, but in a lab, sometimes you have to re-wire the whole experiment, plugging hardware into different outlets and minimising cable lengths, to avoid cross-talk and 'ground noise'. That ground noise is being used as the signal here.

\n

one domain's noise is another domain's data. 

", "rawMarkdown": "", "votes": null }, { "id": "29131", "postDate": "08/21/2013 08:46:24", "content": "

I'm also confused with physical meaning of phase-1 and phase-2 data. Does it show voltage of 60,120,180,240,300,360 Hz harmonics? If so, why the period of them so unstable? And how is it related to the frequencies mentioned above?

\n

Let's take phase-1, 1st harmonics (supposed to be 60 Hz). The period fluctuation histogram of Buffer.LF1V(:,1) real component is in the image.

\n

\n

Suppose my question is silly and I'd be thankful to those who provide some physics behind the terms used in the competition. 

\n

 

\n

P.S. Here gallamine asked very similar question. According to his profile, he's a PhD in electrical engineering but still had some problems with data understanding. In paper [3] there are several references about harmonics but no clear definition of the form they are used in detection problem. 

", "rawMarkdown": "", "votes": null }, { "id": "29193", "postDate": "08/21/2013 20:45:51", "content": "

Hello,

\n

actually I think the periodic term is removed in the complex representation. So the complex representation is not supposed to be periodic.

\n

For example if F is the frequency, a periodic signal u(t) = U.cos(2.pi.F.t + phase) is represented by constant complex C = U.exp(j.phase) and u(t) = Re[C.exp(2.pi.j.F.t)]

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 29097, "author_name": "kiihne", "author_url": "", "post_date": "08/20/2013 19:50:56", "content": "

As I understand it, the high frequency data is a matrix with each line a vector of 4096 values. One vector for each time point. Each vector is calculated by a Fourier Transform of the original noise signals measured by the hardware. Thus, each vector in the HF data is a spectrum of high frequency noise (just like a rainbow is a spectrum of light). Each value in the vector is the magnitude or intensity of the noise at that frequency. The papers suggest that modern power-supplies have a noise signature: a characteristic pattern of high frequency noise intensities, and that these patterns can be used to distinguish between household devices. 

", "votes": null, "replies": [] }, { "id": 29114, "author_name": "royd114539", "author_url": "", "post_date": "08/21/2013 04:09:28", "content": "

Yes, I agree completely. The data comes from the signal in the frequency domain. My second question is asking what the signal actually is. Is it a voltage signal or a power signal? Is it the signal in phase 1, phase 2, both phases measured independently and concatenated into one vector, or the sum across both phases? (Note that the low frequency data is divided into phase 1 and phase 2 signals.)

", "votes": null, "replies": [] }, { "id": 29117, "author_name": "kiihne", "author_url": "", "post_date": "08/21/2013 05:14:03", "content": "

I don't think that the low frequency data is actually 'divided into phase1 and phase2 signals'. If you look at the Buffer data, which is what is actually loaded from the file, it consists of current (I) and voltage (V) on two lines, L1 and L2. Both measurements are at multiple harmonics of the 60Hz line frequency. Each datapoint is a complex number. The HF data is separate. My understanding is that it corresponds to high frequency noise measurements or EMI. The sampling rate needed for these measurements is much higher than the TimeTicks in either L1 or L2. (1MHz, or 1e6/second). The FFT makes the noise data considerably smaller than the raw form. 

\n

Have a good look at the reference from UbiComp, 2010 (ElectriSense: Single-Point Sensing Using EMI for Electrical Event Detection and Classification in the Home by Sidhant Gupta, Matthew S. Reynolds, Shwetak N. Patel). It has a circuit diagram and a full description of the data and analyses tried earlier. 

\n


", "votes": null, "replies": [] }, { "id": 29120, "author_name": "royd114539", "author_url": "", "post_date": "08/21/2013 05:40:07", "content": "

What I meant by 'phases', which seems to be the term this data set uses, is what you're calling 'lines'. (The term appears on this page: https://www.kaggle.com/c/belkin-energy-disaggregation-competition/data) I believe they're usually referred to as 'mains', but phase makes sense since they, ideally, would be perfectly out of phase. So while the low frequency data does have information for lines 1 and 2, the high frequency has no such division explicitly present.

\n

Your comments bring me back to my first question, which is that the ElectriSense sensor could not possibly measure the current through the mains itself.

\n

 

", "votes": null, "replies": [] }, { "id": 29125, "author_name": "kiihne", "author_url": "", "post_date": "08/21/2013 06:39:07", "content": "

Oh. I get it. No, I don't know how they're measuring the current, but I suspect that is part of the Belkin proprietary hardware. 

\n

For the HF measurement:(wikipedia) Electromagnetic_interference. This is primarily about radiated or RF interference, but similar effects happen on power lines.

\n

You might also try searching for 'ground loop' or 'ground loop interference'.  With consumer devices, we don't usually run into trouble, but in a lab, sometimes you have to re-wire the whole experiment, plugging hardware into different outlets and minimising cable lengths, to avoid cross-talk and 'ground noise'. That ground noise is being used as the signal here.

\n

one domain's noise is another domain's data. 

", "votes": null, "replies": [] }, { "id": 29131, "author_name": "", "author_url": "", "post_date": "08/21/2013 08:46:24", "content": "

I'm also confused with physical meaning of phase-1 and phase-2 data. Does it show voltage of 60,120,180,240,300,360 Hz harmonics? If so, why the period of them so unstable? And how is it related to the frequencies mentioned above?

\n

Let's take phase-1, 1st harmonics (supposed to be 60 Hz). The period fluctuation histogram of Buffer.LF1V(:,1) real component is in the image.

\n

\n

Suppose my question is silly and I'd be thankful to those who provide some physics behind the terms used in the competition. 

\n

 

\n

P.S. Here gallamine asked very similar question. According to his profile, he's a PhD in electrical engineering but still had some problems with data understanding. In paper [3] there are several references about harmonics but no clear definition of the form they are used in detection problem. 

", "votes": null, "replies": [] }, { "id": 29193, "author_name": "cbourguignat", "author_url": "", "post_date": "08/21/2013 20:45:51", "content": "

Hello,

\n

actually I think the periodic term is removed in the complex representation. So the complex representation is not supposed to be periodic.

\n

For example if F is the frequency, a periodic signal u(t) = U.cos(2.pi.F.t + phase) is represented by constant complex C = U.exp(j.phase) and u(t) = Re[C.exp(2.pi.j.F.t)]

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "28985": "", "29097": "", "29114": "", "29117": "", "29120": "", "29125": "", "29131": "", "29193": "" }, "source": "meta" }